Knowledge Graphs for Digitized Manuscripts in Jagiellonian Digital Library Application

Grzegorz J. Nalepa; Jan Ignatowicz; Krzysztof Kutt

REVIEW 4 major objections 4 minor 10 references

Knowledge Graphs for Digitized Manuscripts in Jagiellonian Digital Library Application

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that retraining computer vision models on 100 annotated manuscript pages yields accurate detection of text regions, stamps, seals, and decorative features, enabling metadata enrichment and knowledge graph construction.

desk verdict The paper's only empirical claim — retrained models achieve 'considerably accuracies' — is never quantified, leaving the feasibility argument an anecdote. read the letter →

arxiv 2506.03180 v1 pith:GOGJST4B submitted 2025-05-29 cs.DL cs.CV

Jan Ignatowicz , Krzysztof Kutt , Grzegorz J. Nalepa This is my paper

classification cs.DLcs.CV

keywords DigitalHumanitiesImageUnderstandingComputerVisionKnowledgeGraphsDeepLearningmetadataenrichmentdigitizedmanuscriptsculturalheritage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a small, hand-built dataset of 100 annotated pages is enough to adapt general-purpose computer vision models to digitized historical manuscripts, so that libraries can automatically find text regions, paragraphs, headers, stamps, seals, ornaments, and initials. It argues that these detected features can be written into library metadata and then organized into knowledge graphs that link artifacts to external resources like Wikidata and DBpedia. If the claim holds, institutions with limited annotation capacity could enrich their own digitized collections and enable cross-collection searches by visual characteristics. The paper presents this as a work-in-progress feasibility study and does not yet report held-out test accuracy or per-class error rates.

What carries the argument

The mechanism that carries the argument is a two-stage pipeline. In the first stage, general-purpose pre-trained computer vision models are retrained on a 100-image manually annotated dataset, with categories including paragraphs, stains, stamps, descriptions, signs, signatures, images, ornaments, initials, and headers; this retraining is what adapts the models to the visual language of manuscripts and incunabula. In the second stage, the detections are converted into metadata fields, and a modular ontology—structured around physical attributes, visual features, and provenance—organizes those fields into a knowledge graph using Linked Data principles. The retrained detector supplies the content, and the ontology supplies the queryable structure; the paper's claim is that both stages are feasible with the small dataset and off-the-shelf models.

What would settle it

Build a held-out test set by sampling pages randomly across the collection's different scripts, layouts, damage levels, and centuries, run the retrained models on it, and compare per-class detection accuracy (for example mean average precision for stamps, seals, headers, and ornaments) against accuracy on the training images. If the held-out accuracy is near chance or far below the training accuracy, the feasibility claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that retraining pre-trained models—YOLOv11, Detectron2, DeepLabv3, U-Net, and HRNet—on 100 manually annotated images from the Jagiellonian Digital Library gives 'considerably accuracies' in detecting manuscript-specific objects such as paragraphs, headers, stamps, seals, ornaments, and illuminated initials. The retrained models were then used to analyze digitized manuscripts and to enrich each document's metadata with the position, size, and type of detected text sections, plus the presence and location of stains, signs, and decorative features. The paper claims this enriched metadata is a suitable basis for constructing knowledge graphs through a modular ontology covering physical attributes, visual features, and provenance, connected through Linked Data. The paper does not quantify the accuracies or provide a test set, so the discovery is asserted at the level of feasibility rather than measured performance.

Load-bearing premise

The load-bearing premise is that 100 manually chosen annotated pages represent the full variety of the Jagiellonian collection's scripts, layouts, damage states, and historical periods, so the retrained models will work on pages that were not in the training sample.

Editorial extensions

If this is right

Libraries holding only a few dozen annotated pages can adapt general-purpose vision models to their digitized collections without training a detector from scratch.
Detected visual elements can be written into library metadata, so search and comparison can use physical and visual characteristics rather than title, author, and keywords alone.
The enriched metadata can feed knowledge graphs that link artifacts to Wikidata and DBpedia, enabling queries that track recurring seals or the evolution of decorative styles across a collection.
The same pipeline can be applied to growing datasets as models improve, which positions the approach to scale beyond the initial corpus.
By connecting enriched records through linked data, cross-collection analysis becomes possible even when institutions follow differing metadata standards.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The feasibility claim would gain strength from per-class results, because rare classes such as seals and stamps are the ones most likely to decide whether cross-collection searches by visual feature actually work, and the paper reports only an overall accuracy statement.
If the 100-image recipe transfers to other libraries, metadata enrichment could become a shared service in which each institution annotates a small sample and shares fine-tuned models rather than building a full computer-vision pipeline from scratch.
A natural test the paper does not run is a retrieval experiment: ask historians to find manuscripts matching a given ornament or seal using the enriched knowledge graph, then compare success rates against the current title-author-keyword metadata.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The real news in this paper is one sentence: retraining off-the-shelf object detectors on 100 annotated manuscript pages produced "considerably accuracies." That could be an interesting data point for digital libraries, but the paper never gives a number. No mAP, no precision/recall, no test set, no baseline comparison. So the central empirical claim is an anecdote.

What the paper does well is frame a legitimate problem: digitized manuscripts come with catalog metadata but almost nothing about visual content. The proposed workflow — detect text regions, stamps, seals, ornaments, then attach those features to metadata and feed a knowledge graph — is sensible and matches how GLAM institutions are moving. The 100-image annotated dataset, small as it is, is a reasonable first step, and the category list (paragraphs, stains, stamps, initials, etc.) is domain-appropriate. The knowledge graph section is standard Linked Data practice, but it's a proposal, not an implementation.

The soft spot is not the small dataset; it's that the only reported result is unverified. "Accuracies fell below acceptable thresholds" for the pretrained models, then "considerably accuracies" after retraining — those are the before/after numbers, in prose. There is no way to check whether the retrained models actually work, which makes the feasibility argument unfalsifiable from the manuscript. The representativeness of 100 hand-picked pages is a real secondary concern, but it only matters once the accuracy claim is actually measured. Also, the enriched metadata and KG remain qualitative descriptions.

This looks like a short work-in-progress writeup that might have been a poster or a workshop abstract. For a research paper, the evaluation gap is too large. The authors should either release the annotated dataset and report standard detection metrics with a train/test split, or position the paper explicitly as a position statement without the empirical claim.

Who is this for? Practitioners in digital libraries who want a quick sketch of how off-the-shelf CV can be pointed at manuscripts. They'll get a workflow and a list of categories, but no evidence to adopt it. For a serious referee, I'd want at least a test set and numbers before sending it out.

Recommendation: don't accept as is; desk reject or send back for major revision with the requirement of quantitative evaluation. Engaging with the authors' next version could be worth it, because the problem is real and the dataset, if released, would be a useful contribution.

Referee Report

4 major / 4 minor

Summary. The paper reports work-in-progress on enriching metadata for digitized manuscripts and incunabula from the Jagiellonian Digital Library. The authors state that off-the-shelf detection models (YOLOv11, Detectron2, DeepLabv3, U-Net, HRNet) perform poorly on historical manuscripts, and that retraining them on a small dataset of 100 manually annotated images led to 'considerably accuracies'. They then describe how the detected features (paragraphs, stamps, seals, initials, etc.) are added to metadata, and outline a plan for constructing knowledge graphs using ontologies and Linked Data. The paper is primarily a proposal: the knowledge graph is not implemented, and no quantitative evaluation of the detection results is reported.

Significance. If the central empirical claim were properly supported, the paper would make a useful contribution by showing that general-purpose vision models can be adapted to digitized manuscripts with a very small annotation budget, thereby enabling automated metadata enrichment for cultural heritage collections. However, as written, the paper contains no measurable evidence: no test set, no detection metrics, no comparison with the pretrained baselines, and no analysis of the annotated dataset. The knowledge graph section is a plan rather than a deployed system. The contribution is therefore currently an anecdote plus an architecture sketch, and the claimed feasibility is not established. The possible value of the underlying idea and dataset should be acknowledged, but the manuscript in its present form does not meet the evidentiary standard for a research paper.

major comments (4)

[§2.1] The central claim is made in a single unquantified sentence: 'With the use of this dataset, we retrained the pre-trained models and eventually achieved considerably accuracies in the object detection task.' No evaluation protocol is given: there is no train/test split, no mAP, IoU, precision, recall, confusion matrix, error bars, or any comparison with the earlier poor performance of the same models. Without these numbers, the reader cannot verify that the retraining was successful, cannot compare the approach to existing methods, and cannot assess whether the claimed adaptation to historical manuscripts actually occurred. This is the only empirical result in the paper, and it is untestable as reported.
[§2.2] The metadata enrichment section claims that the retrained models 'detected text regions, classified paragraphs and headers, and highlighted decorative elements', and that these detections were added to metadata. These claims rest entirely on the detection quality asserted in Section 2.1, which is never measured. Additionally, no examples of the enriched metadata are shown, no validation of the categories is provided, and the class list (paragraphs, stains, stamps, descriptions, signs, signatures, images, ornaments, initials, headers) is given without any explanation of how these categories were defined or how reliable their detection is. The usefulness of the downstream metadata enrichment therefore cannot be assessed.
[§3] The knowledge graph construction is described only in future tense and as a proposal ('proposes a modular ontology', 'These connected through Linked Data principles'). No ontology is defined, no instances are created, no SPARQL endpoints or example queries are given, and no demonstration of connection to DBpedia or Wikidata is shown. Given that the title promises 'Knowledge Graphs for Digitized Manuscripts ... Application', the absence of any implemented knowledge graph or even a concrete schema means the paper does not deliver on its advertised contribution.
[§2.1] The described dataset of 100 manually annotated images is not characterized. There is no information about how the images were selected, whether they span different Scripts, layouts, damage states, or historical periods, what the distribution of classes is, how many annotators were involved, or what the inter-annotator agreement was. Since the paper argues that retraining on this dataset makes the models feasible for the entire Jagiellonian collection, the representativeness of this small sample is load-bearing. As it stands, the assertion that the dataset 'captures elements such as text regions, stamps, seals, and decorative features' is not enough to support generalization to the full heterogeneity of the collection.

minor comments (4)

[Throughout] The paper contains numerous language and typographical errors that should be corrected, e.g., 'creates' for 'create' in the abstract, 'sttributes' for 'attributes' in Section 3, 'Is also' for 'It also' in Section 4, and 'what contribute' for 'which contribute' in Section 2.2.
[§2.1] The phrase 'considerably accuracies' is grammatically incorrect and ambiguous; the authors should specify which metric improved and by how much.
[§2.2] The reference to 'Figure 1' shows only a single annotated image; a small multi-panel figure showing several examples across different classes would help the reader understand the annotation categories.
[§5] The paper labels itself as 'work-in-progress' and concludes with future work on expanding the dataset and automation of ontology generation; this framing is honest but reinforces that the presented results are preliminary.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the workflow is a proposed pipeline whose empirical claim, though unquantified, is not assumed by construction.

full rationale

The paper's chain is a proposed pipeline: annotate 100 images, retrain off-the-shelf CV models, apply them to digitized manuscripts, add the detections to metadata, and propose a modular-ontology KG over that enriched metadata. None of these steps is equivalent to its own input by construction. The retraining uses manual annotations that are external to the paper, and the 'considerably accuracies' statement is an empirical claim; the absence of a test set, metrics, or baselines makes it untestable, but untestability is an evidence/correctness issue, not circularity. The KG construction is explicitly prospective ('the next step is to build', 'Our approach proposes'), so there is no implemented output that is fed back as an input. There are no load-bearing self-citations: references [1]-[10] are standard external systems (YOLOv11, Detectron2, DeepLabv3, U-Net, HRNet, DBpedia, Wikidata, OAI-PMH) or the institutional library site, and no prior work by the same authors is invoked as a uniqueness theorem or forced ansatz. The category list and ontology modules are proposed schema definitions rather than predictions fitted from data. Therefore the appropriate critique is that the sole empirical result is an anecdote without evaluation, not that the derivation is circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's feasibility claim rests on the representativeness and correctness of a 100-image manually annotated dataset and on the unstated suitability of its 10 categories. It also assumes that the proposed modular ontology is constructible and useful, but no part of that ontology is specified. No test set, error rates, or data release are provided, so all assumptions remain unchecked.

free parameters (3)

Retrained model weights = not reported
The claimed detection accuracy depends on weights obtained by retraining YOLOv11, Detectron2, DeepLabv3, U-Net, and/or HRNet on the 100-image dataset (Section 2.1). The weights are not published.
Training hyperparameters = not reported
No learning rates, epochs, augmentation, or optimizer settings are given; these affect the claimed accuracy and are not reproducible.
Annotation category set = 10 classes
The set of detected classes (paragraphs, stains, stamps, descriptions, signs, signatures, images, ornaments, initials, headers) is chosen by the authors without evidence of optimality (Section 2.2).

assumptions (4)

domain assumption The 100 manually annotated images are representative of the Jagiellonian collection's full visual diversity.
Section 2.1 asserts the dataset supports retraining, but no test set, sampling strategy, or distribution analysis is provided.
domain assumption Manual annotations are correct and consistently labeled.
No annotation protocol, annotator qualifications, or inter-annotator agreement is described (Section 2.1).
domain assumption The 10 chosen categories capture the visual features that matter for scholarly metadata enrichment.
Section 2.2 lists the categories, but there is no evidence these classes match the queries researchers would run.
domain assumption A modular ontology can be built to connect detected features, catalog metadata, and external resources like Wikidata.
Section 3 proposes this ontology but provides no schema, axioms, or implementation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Knowledge Graphs for Digitized Manuscripts in Jagiellonian Digital Library Application." pith.science (2026). https://pith.science/paper/GOGJST4B

@misc{pith2026250603180,
  author       = {Pith},
  title        = {Pith review of: Knowledge Graphs for Digitized Manuscripts in Jagiellonian Digital Library Application},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GOGJST4B}},
  note         = {Machine review of arXiv:2506.03180}
}

read the original abstract

Digitizing cultural heritage collections has become crucial for preservation of historical artifacts and enhancing their availability to the wider public. Galleries, libraries, archives and museums (GLAM institutions) are actively digitizing their holdings and creates extensive digital collections. Those collections are often enriched with metadata describing items but not exactly their contents. The Jagiellonian Digital Library, standing as a good example of such an effort, offers datasets accessible through protocols like OAI-PMH. Despite these improvements, metadata completeness and standardization continue to pose substantial obstacles, limiting the searchability and potential connections between collections. To deal with these challenges, we explore an integrated methodology of computer vision (CV), artificial intelligence (AI), and semantic web technologies to enrich metadata and construct knowledge graphs for digitized manuscripts and incunabula.

Figures

Figures reproduced from arXiv: 2506.03180 by the authors.

**Figure 1.** An example image from the created dataset. The image highlights the labeled classes: stamp, initial, header, ornament, and paragraph. 3 Knowledge Graph Construction With enriched metadata, the next step is to build knowledge graphs that semantically represent each artifact’s attributes and relationships. For a manuscript, this step includes linking detected features (e.g., ”Stamp X belongs to section Y”). It connec… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

10 extracted references · 5 canonical work pages

[1]

In: The Semantic Web, 6th International Semantic Web Conference (ISWC)

Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: Dbpedia: A nucleus for a web of open data. In: The Semantic Web, 6th International Semantic Web Conference (ISWC). pp. 722–735. Springer (2007). https://doi.org/10.1007/978-3-540-76298-0˙52, https://link.springer. com/chapter/10.1007/978-3-540-76298-0_52

doi:10.1007/978-3-540-76298-0 2007
[2]

https://www.w3.org/ DesignIssues/LinkedData.html (2006)

Berners-Lee, T.: Linked data: Design principles for the web. https://www.w3.org/ DesignIssues/LinkedData.html (2006)

work page 2006
[3]

In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), https://arxiv.org/ abs/1706.05587

Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolu- tion for semantic image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), https://arxiv.org/ abs/1706.05587

arXiv 2017
[4]

https://jbc.bj.uj.edu.pl/ dlibra (2025), [Online; accessed 24-April-2025]

Jagiellonian University: Jagiellonian digital library. https://jbc.bj.uj.edu.pl/ dlibra (2025), [Online; accessed 24-April-2025]

work page 2025
[5]

https://github

Jocher, G., Team, U.: Yolov11: You only look once, version 11. https://github. com/ultralytics/yolov11 (2024) Knowledge Graphs for Digitized Manuscripts in JDL 5

work page 2024
[6]

https://www.openarchives.org/OAI/openarchivesprotocol

Open Archives Initiative: The open archives initiative protocol for metadata har- vesting (oai-pmh). https://www.openarchives.org/OAI/openarchivesprotocol. html (2002)

work page 2002
[7]

Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation (2015)

2015
[8]

Com- munications of the ACM 57(10), 78–85 (2014)

Vrandeˇ ci´ c, D., Kr¨ otzsch, M.: Wikidata: A free collaborative knowledge base. Com- munications of the ACM 57(10), 78–85 (2014). https://doi.org/10.1145/2629489, https://dl.acm.org/doi/10.1145/2629489

doi:10.1145/2629489 2014

Show all 10 references

[9]

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 43(10), 3349–3364 (2020)

Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., Liu, W., Xiao, B.: Deep high- resolution representation learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 43(10), 3349–3364 (2...

2020
[10]

https:// github.com/facebookresearch/detectron2 (2019)

Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. https:// github.com/facebookresearch/detectron2 (2019)

2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

In: The Semantic Web, 6th International Semantic Web Conference (ISWC)

Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: Dbpedia: A nucleus for a web of open data. In: The Semantic Web, 6th International Semantic Web Conference (ISWC). pp. 722–735. Springer (2007). https://doi.org/10.1007/978-3-540-76298-0˙52, https://link.springer. com/chapter/10.1007/978-3-540-76298-0_52

doi:10.1007/978-3-540-76298-0 2007

[2] [2]

https://www.w3.org/ DesignIssues/LinkedData.html (2006)

Berners-Lee, T.: Linked data: Design principles for the web. https://www.w3.org/ DesignIssues/LinkedData.html (2006)

work page 2006

[3] [3]

In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), https://arxiv.org/ abs/1706.05587

Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolu- tion for semantic image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), https://arxiv.org/ abs/1706.05587

arXiv 2017

[4] [4]

https://jbc.bj.uj.edu.pl/ dlibra (2025), [Online; accessed 24-April-2025]

Jagiellonian University: Jagiellonian digital library. https://jbc.bj.uj.edu.pl/ dlibra (2025), [Online; accessed 24-April-2025]

work page 2025

[5] [5]

https://github

Jocher, G., Team, U.: Yolov11: You only look once, version 11. https://github. com/ultralytics/yolov11 (2024) Knowledge Graphs for Digitized Manuscripts in JDL 5

work page 2024

[6] [6]

https://www.openarchives.org/OAI/openarchivesprotocol

Open Archives Initiative: The open archives initiative protocol for metadata har- vesting (oai-pmh). https://www.openarchives.org/OAI/openarchivesprotocol. html (2002)

work page 2002

[7] [7]

Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation (2015)

2015

[8] [8]

Com- munications of the ACM 57(10), 78–85 (2014)

Vrandeˇ ci´ c, D., Kr¨ otzsch, M.: Wikidata: A free collaborative knowledge base. Com- munications of the ACM 57(10), 78–85 (2014). https://doi.org/10.1145/2629489, https://dl.acm.org/doi/10.1145/2629489

doi:10.1145/2629489 2014