Pith. sign in

REVIEW 4 major objections 5 minor 28 references

Matching and Linking Entries in Historical Swedish Encyclopedias

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Between its first edition (1876–1899) and second (1904–1926), Nordisk familjebok's geographic entries shifted measurably away from Europe.

desk verdict Useful, honest pipeline paper whose headline geographic claim lacks significance testing and rests on an unverified assumption of unbiased Wikidata linking. read the letter →

arxiv 2507.01170 v1 pith:OX6QOJFX submitted 2025-07-01 cs.CL

Simon Börjesson , Erik Ersmark , Pierre Nugues This is my paper · ORCID
classification cs.CL
keywords NordiskfamiljebokencyclopediaentriesentrysegmentationsemanticembeddingsWikidatalinkinggeographicdistributionhistoricalSwedishtexteditionmatching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the geographic focus of a nation's most influential encyclopedia changed across editions, and it answers with a measured yes. Using digitized text of Nordisk familjebok's first two editions, the authors segment the raw text into individual entries, classify which entries are locations, match corresponding entries between editions, and link each location entry to coordinates in the Wikidata knowledge graph. They find a small but statistically significant shift in the geographic distribution of entries away from Europe and toward North America, Africa, Asia, Australia, and northern Scandinavia between the 1876–1899 and 1904–1926 editions. This matters because it converts an anecdotal historical narrative — that the First World War and new global powers reshaped Swedish intellectual attention — into a reproducible, quantitative observation.

What carries the argument

The argument is carried by a four-stage linking pipeline. Entry boundaries are recovered from the digitized OCR text by a three-step segmenter: paragraphs opening with a bold tag, paragraphs whose start matches an index headword with a length-relative Levenshtein distance below 0.15, and otherwise a CLD3-style binary classifier. A location classifier built from the [CLS] hidden states of a Swedish BERT model (KB-BERT) with a logistic head selects geographic entries. The selected entries are embedded with a Swedish sentence-transformer model (KB-SBERT) and matched across editions by greedy nearest-neighbor search in a vector database with a cosine threshold of 0.9. For each entry, the pipeline queries Wikidata, a collaborative knowledge graph, encodes the entry and the top five candidate texts, and keeps the candidate above a cosine threshold of 0.6, whose coordinate property (P625) supplies the latitude and longitude. The continent and country distribution of these coordinates between editions is what the geographic-shift claim is read off.

What would settle it

Recompute the continent distribution using only the entries that the paper's own 25-kilometer accuracy metric accepts; if the shift away from Europe disappears or reverses in that high-confidence subset, the reported shift is an artifact of geographically biased linking rather than an editorial change.

Watch

Extended reading notes

Core claim

This paper's central discovery is that the second edition of Nordisk familjebok (1904–1926) contains a measurably more global geographic entry distribution than the first edition (1876–1899). By classifying roughly 22% of entries as locations, matching the two editions' entries, and linking the locations to Wikidata coordinates, the authors observe a shift in the continental distribution of entries: Europe's share decreases while North America, Africa, Asia, Australia, and northern Scandinavia increase. The paper interprets this shift as confirming the influence of the First World War and the rise of new powers on Swedish reference knowledge, and it notes that because the second edition has more location entries overall, a relative decline for European countries does not require an absolute decline in their coverage.

Load-bearing premise

The distributional conclusion rests on the assumption that Wikidata linking succeeds or fails about as often for European and non-European places, so the measured shift is not a by-product of easier linking for some continents.

Editorial extensions

If this is right

  • The second edition of Nordisk familjebok has a measurably more global distribution of geographic entries than the first, with Europe's share falling and North America, Africa, Asia, Australia, and northern Scandinavia rising.
  • The matched-entry dataset can support cross-edition search systems, so a query could return content from both editions at once rather than from a single volume.
  • Because the second edition contains more locations overall, a relative decline in Europe's share does not imply fewer European entries in absolute terms.
  • The public code and data allow the same linking pipeline to be applied to other historical encyclopedias or to later editions of Nordisk familjebok.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper does not break the shift down by volume within the long second edition, so the same data could date the change more precisely if the shift is concentrated in volumes written during or after the First World War.
  • Editorial inference: a testable extension would replace continent-level counts with per-article first-mention analysis, which could separate the addition of new non-European articles from the rewriting of existing ones.
  • Editorial inference: if Wikidata coverage is better for European places, the observed direction of bias would tend to inflate the shift, so measuring linking success on a manually verified stratified sample would show whether the claim survives.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a computational pipeline for comparing the first two editions of the Swedish encyclopedia Nordisk familjebok (1876–1899 and 1904–1926). The authors scrape OCR text from Project Runeberg, segment it into entries, filter out cross-references, classify entries as locations using a KB-BERT-based classifier, match location entries across the two editions with sentence embeddings and a cosine threshold, and link them to Wikidata items with coordinates. The resulting linked location sets are then used to compare continental distributions between editions. The paper reports a 'small but significant shift' in geographic focus away from Europe and toward North America, Africa, Asia, Australia, and northern Scandinavia, which they interpret as reflecting the influence of the First World War and global industrialization. The pipeline is evaluated on validation sets of 25–200 entries, with the weakest component being Wikidata linking (within-25-km F1 of 0.69 for the first edition and 0.54 for the second).

Significance. If the claimed geographic shift is real, this would be a valuable quantitative contribution to digital humanities and historical encyclopedia research, and the released code and data would enable replication and extension. The paper's main strength is the transparent description of a full pipeline, from raw OCR to interpretable geographic trends, with component-wise evaluation and public GitHub resources. However, the central interpretive claim rests on an unverified and, on its face, questionable assumption about the geographical neutrality of Wikidata linking errors. The paper explicitly acknowledges its small evaluation sets and the poor linking quality but still uses the word 'significant' without any statistical test. Because the headline result depends on the distribution of linked locations, and because that distribution could be systematically distorted by differential linking success across continents, the conclusion is not currently supported by the evidence presented. The paper is more convincing as a description of a reusable pipeline than as a demonstration of a historical shift.

major comments (4)
  1. [Section 5.5 and Table 3] The central claim of a geographic shift between editions depends on the assumption, stated in Section 5.5, that Wikidata linking errors are approximately uniformly distributed geographically: 'it seems reasonable to assume that the overall distribution of locations remains roughly the same.' Table 3 shows that only 40–64% of locations are linked within 25 km of the correct coordinates, so a majority of links are either missing or wrong at kilometer scale. If unlinked or incorrectly linked entries are not uniformly distributed across continents, the observed proportional decline of Europe and rise of other regions could be entirely an artifact of differential linking success. The paper does not compare the continent distribution of successfully linked versus unlinked/incorrectly linked entries, nor does it provide any sensitivity analysis. This assumption is load-bearing, and the current text provides no evidence for it.
  2. [Abstract and Section 6.2] The abstract and Section 6.2 use the word 'significant' in a statistical sense ('a small but significant shift'), but the paper reports no confidence intervals, hypothesis tests, or effect-size measures. The location counts per continent are derived from a pipeline with substantial component errors (e.g., location-classifier F1 ≈ 0.90, entry-matching F1 ≈ 0.83, Wikidata linking within-25-km F1 between 0.54 and 0.69). To support the word 'significant', the authors should compare the two editions with a formal test on the continent proportions, or at least provide error bars that propagate the estimated component uncertainties. Without this, the statement overreaches the evidence.
  3. [Sections 4.2 and 4.3.2] The matching threshold (cosine 0.9) and the Wikidata-linking threshold (cosine 0.6) are fitted on validation sets of 200 and 25 entries per edition, respectively. The matched and linked sets that feed the geographic analysis are therefore not independent of these threshold choices. The paper does not assess how sensitive the continental distributions in Figure 2 are to plausible variations of these thresholds. For example, raising or lowering the linking threshold by 0.05 could change which of the first five Wikidata candidates is accepted, and that could change the continent counts disproportionately for sparsely covered regions. A robustness analysis varying both thresholds and reporting the resulting shift would be needed to show that the observed trend is not an artifact of threshold fitting on small sets.
  4. [Table 3 and Section 5.4] The evaluation of entry matching, which is a prerequisite for the removed/added-entry analysis, is based on a validation set used to select the threshold, and the reported F1 of 0.83 is close to the headword baseline of 0.76. The paper states that the improvement is 'not significant,' and no statistical test is provided even there. Given that the pipeline's downstream geographic analysis depends on the set of matched location entries, the small evaluation sets and the lack of significance testing undermine the precision of any claim about which entries were 'added' or 'removed' between editions. The authors acknowledge this in the Limitations section, but the abstract does not reflect this uncertainty.
minor comments (5)
  1. [Section 3.2.2] The relative Levenshtein threshold of 0.15 was set through 'manual testing' without describing the test set or the criterion used; please state how many examples were examined and what error rate was deemed acceptable.
  2. [Section 4.1] The location classifier is trained on 200 manually annotated entries, but the annotation procedure (e.g., who annotated, whether disagreements were resolved, inter-annotator agreement) is not described; this would help readers judge the quality of the ground truth.
  3. [Table 3] The table footnotes are inconsistent: the 'Entry classifier' row reports a validation set of 4 samples, which is too small to be meaningful, and the row is marked with '*' for both editions without explaining that the same model was evaluated only on first-edition data.
  4. [Figure 2c] The caption says 'percentage unit increase' but the figure shows only a list of countries without numerical values; define the metric and provide the underlying numbers, otherwise the reader cannot assess the magnitude of the change.
  5. [Section 5.4] The sentence 'albeit not significantly' appears without a supporting statistical test; if the authors intend a statistical claim, they should report the test, and if not, the phrase should be removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the geographic-shift finding is a descriptive pipeline output, and the acknowledged linking bias is a robustness limitation, not a circular step.

full rationale

The paper does not define its target quantities in terms of its outputs, nor does it fit a parameter to the quantity it later presents as a finding. The matching threshold (cosine 0.9) and Wikidata-linking threshold (cosine 0.6) are tuned on small validation sets, but the reported geographic shift is computed from the full linked corpus; it is not a validation metric, so there is no fitted-input-called-prediction reduction. The location classifier is trained on 200 manually annotated entries (Section 4.1), not on the bold-tag heuristic used for entry segmentation (Section 3.2.3), so the reader's suggestion that the same structural heuristic is reused is not supported by the text. The only load-bearing empirical bridge is the assumption in Section 5.5 that incomplete Wikidata linking leaves 'the overall distribution of locations roughly the same,' but this is an unverified assumption about missing data, not a definitional equivalence. Ahlin et al. (2024) is cited for text truncation and for comparison of entry counts and location ratios; those citations are not used to justify the central distributional claim and are not load-bearing. The Limitations section explicitly acknowledges the small validation sets, which affects reliability but does not make the derivation circular. The central claim is a descriptive statistic over pipeline outputs and is therefore not circular, though its robustness is limited by the acknowledged low linking recall.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests mainly on the quality and unbiasedness of the classification and linking steps. The thresholds that define final matches and links are fitted on small validation sets, and the linking error assumption is explicit.

free parameters (4)
  • Entry matching cosine threshold = 0.9
    Tuned to maximize F1 on a manually annotated set of 200 entries (Section 4.2); then applied to produce all edition pairs.
  • Wikidata linking cosine threshold = 0.6
    Chosen via evaluation on two test sets of 25 random locations, one per edition (Section 4.3.2).
  • Relative Levenshtein distance threshold = 0.15
    Set through manual testing to match index headwords to text (Section 3.2.2).
  • Entry text truncation length = 200 characters
    Used to limit location classifier input, following Ahlin et al. (2024); not fitted but chosen by hand and influences classification and matching.
assumptions (6)
  • domain assumption Bold-tagged paragraphs are valid entries
    Used as ground truth for entry classification training and as the primary segmentation strategy (Sections 3.2.1 and 3.2.3). If OCR often loses the bold markup, this assumption undercounts entries.
  • domain assumption In a volume, every entry begins with the same capital letter, so paragraphs starting with other capital letters are non-entries
    Used to generate negative training examples for the entry classifier (Section 3.2.3). OCR errors or dropped headwords could violate this assumption.
  • domain assumption KB-SBERT embeddings trained on modern Swedish transfer sufficiently well to 19th-early 20th century encyclopedic text
    The authors rely on these embeddings for both entry matching and Wikidata linking (Sections 4.2 and 4.3.2) and note the language difference as a limitation.
  • ad hoc to paper Wikidata linking errors are approximately uniformly distributed geographically
    Stated in Section 5.5 as 'reasonable to assume that the overall distribution of locations remains roughly the same'; this is load-bearing for the geographic shift claim.
  • domain assumption Cross-references are entries shorter than 60 characters containing ' Se '
    Used to filter cross-references (Section 3.3). Longer cross-references are left, and some short ones may be misclassified.
  • ad hoc to paper Cosine similarity thresholds fitted on small validation sets generalize to the full corpus
    Used to decide final matches and links (Sections 4.2 and 4.3.2), without a held-out test set.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Matching and Linking Entries in Historical Swedish Encyclopedias." pith.science (2026). https://pith.science/paper/OX6QOJFX

@misc{pith2026250701170,
  author       = {Pith},
  title        = {Pith review of: Matching and Linking Entries in Historical Swedish Encyclopedias},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OX6QOJFX}},
  note         = {Machine review of arXiv:2507.01170}
}
read the original abstract

The \textit{Nordisk familjebok} is a Swedish encyclopedia from the 19th and 20th centuries. It was written by a team of experts and aimed to be an intellectual reference, stressing precision and accuracy. This encyclopedia had four main editions remarkable by their size, ranging from 20 to 38 volumes. As a consequence, the \textit{Nordisk familjebok} had a considerable influence in universities, schools, the media, and society overall. As new editions were released, the selection of entries and their content evolved, reflecting intellectual changes in Sweden. In this paper, we used digitized versions from \textit{Project Runeberg}. We first resegmented the raw text into entries and matched pairs of entries between the first and second editions using semantic sentence embeddings. We then extracted the geographical entries from both editions using a transformer-based classifier and linked them to Wikidata. This enabled us to identify geographic trends and possible shifts between the first and second editions, written between 1876-1899 and 1904-1926, respectively. Interpreting the results, we observe a small but significant shift in geographic focus away from Europe and towards North America, Africa, Asia, Australia, and northern Scandinavia from the first to the second edition, confirming the influence of the First World War and the rise of new powers. The code and data are available on GitHub at https://github.com/sibbo/nordisk-familjebok.

Figures

Figures reproduced from arXiv: 2507.01170 by the authors.

Figure 1
Figure 1. Overview of the pipeline. beginning with a bold tag is almost certainly a valid entry, we used these paragraphs as ground truth for entries, removing bold tags in the process. Addi￾tionally, we used the fact that an encyclopedia is alphabetically ordered to find ground truth for non￾entries. For example, in a volume, where all entries begin with the letter K, a paragraph starting with any other capital letter is a n… view at source ↗
Figure 2
Figure 2. Location-related statistics from both editions. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Geographic distribution of locations in both editions. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 20 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Axel Ahlin, Alfred Myrne Bl der, and Pierre Nugues. 2024. https://aclanthology.org/2024.lrec-main.962 Mapping the past: Geographically linking an early 20th century S wedish encyclopedia with W ikidata . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 11040...

    work page 2024

  4. [4]

    Lars Aronsson. 2003. https://runeberg.org/nf/#preface Preface to the digital facsimile edition . Last accessed 2024-06-07

    work page 2003

  5. [5]

    Lars Aronsson. 2023. https://runeberg.org/admin/ About P roject R uneberg . Last accessed 2024-06-03

    work page 2023

  6. [6]

    Jakob Christensson. 2005. I encyklopediernas trollkrets: Om Bernhard Meijer och Nordisk familjebok. Biblis, 2005(32):32--49

    work page 2005

  7. [7]

    Brigitte Cordier. 1965. https://aclanthology.org/C65-1003 Factor-analysis of correspondences . In COLING 1965

    work page 1965

  8. [8]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

    doi:10.18653/v1/n19-1423 2019

Show all 28 references
  1. [9]

    Harold W Kuhn. 1955. The Hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83--97

    1955

  2. [10]

    Sanjay Kukreja, Tarun Kumar, Vishal Bharate, Amit Purohit, Abhijit Dasgupta, and Debashis Guha. 2023. https://doi.org/10.1109/IWAIIP58158.2023.10462847 Vector databases and vector embeddings-review . In 2023 International Workshop on Artificial Intelligence and Image Processin...

    2023

  3. [11]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. https://arxiv.org/abs/2405.17428 NV-Embed : Improved techniques for training LLM s as generalist embedding models . Preprint, arXiv:2405.17428

    2024 arXiv

  4. [12]

    Vladimir Levenshtein. 1966. Binary Codes Capable of Correcting Deletions, Insertions and Reversals . Soviet Physics Doklady, 10:707--710

    1966

  5. [13]

    Lewis, Yiming Yang, Tony G

    David D. Lewis, Yiming Yang, Tony G. Rose, and Fan Li. 2004. RCV1 : A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361--397

    2004

  6. [14]

    Martin Malmsten, Love B \"o rjeson, and Chris Haffenden. 2020. https://doi.org/10.48550/arXiv.2007.01658 Playing with words at the N ational L ibrary of S weden--making a S wedish BERT . arXiv preprint arXiv:2007.01658

    2020 doi arXiv

  7. [15]

    Rui Meng, Ye Liu, Shafiq Rayhan Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. 2024. https://huggingface.co/Salesforce/SFR-Embedding-2_R SFR-Embedding-2 : Advanced text embedding with multi-stage training

    2024

  8. [16]

    Pierre Nugues. 2022. https://aclanthology.org/2022.lrec-1.272 Connecting a F rench dictionary from the beginning of the 20th century to W ikidata . In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2548--2555, Marseille, France. European Lang...

    2022

  9. [17]

    Gertrud Pettersson. 2005. Svenska språket under sjuhundra år: En historia om svenskan och dess utforskande, 2nd edition. Studentlitteratur, Lund

    2005

  10. [18]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.18653/v1/D19-1410 Sentence- BERT : Sentence embeddings using S iamese BERT -networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference...

    2019 doi

  11. [19]

    Faton Rekathati. 2023. https://kb-labb.github.io/posts/2023-01-16-sentence-transformer-20/ The KBL ab blog: S wedish sentence transformer 2.0 . Last accessed 2024-06-05

    2023

  12. [20]

    Abdul Lathif Fathima Shanaz and Roshan G. Ragel. 2021. https://doi.org/10.1109/ICIAfS52090.2021.9606139 Wikidata based person entity linking in news articles . In 2021 10th International Conference on Information and Automation for Sustainability (ICIAfS), pages 66--70

    2021

  13. [21]

    Maria Simonsen. 2016. Den skandinaviske encyklopædi: Udgivelse og udformning af Nordisk familjebok & Salmonsens konversationslexikon . Centrum för Öresundsstudier (Print), 37. Makadam i samarbete med Centrum för Öresundsstudier vid Lunds universitet, Göteborg ; Stockholm

    2016

  14. [22]

    Michael Snape. 2018. https://doi.org/doi:10.1017/S0022046917000616 Anglicanism and interventionism: Bishop B rent, the U nited S tates, and the B ritish empire in the F irst W orld W ar . The Journal of Ecclesiastical History, 69(2):300--325

    2018 doi

  15. [23]

    Karen Sp\" a rck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28(1):11–21

    1972

  16. [24]

    Samuel Sundvall. 2023. https://doi.org/doi:10.1017/S0956793323000092 Migration and decentralised industrialisation: The development of rural migration in northern S weden (1850--1950) . Rural History, pages 1--20

    2023 doi

  17. [25]

    Claudia Thomson. 1938. N orway's industrialization. Economic Geography, 14(4):372--380

    1938

  18. [26]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . Advances in Neura...

    2017

  19. [27]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/W18-5446 GLUE : A multi-task benchmark and analysis platform for natural language understanding . In Proceedings of the 2018 EMNLP Workshop B lackbox NLP : A...

    2018 doi

  20. [28]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. https://doi.org/10.1145/3626772.3657878 C-Pack : Packed resources for general Chinese embeddings . In Proceedings of the 47th International ACM SIGIR Conference on Research and Develo...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.