Pith. sign in

REVIEW 3 major objections 5 minor 73 references

Musical Heritage Historical Entity Linking

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports that a retrieval-based linker constrained by knowledge-graph time and type filters, with a score-based NIL heuristic, outperforms all tested specialized linkers and two large language models on historical…

desk verdict The dataset is the contribution; the model-vs-LLM claim is post-selected on test-set thresholds and runs with gold type information, so the comparison overstates the win. read the letter →

arxiv 2502.09168 v1 pith:7D5KFXUA submitted 2025-02-13 cs.CL

Arianna Graciotti , Nicolas Lazzari , Valentina Presutti , Rocco Tripodi This is my paper · ORCID
classification cs.CL
keywords entitylinkinghistoricaldocumentsNILpredictionknowledgegraphconstraintsretrieval-basedmodelsgametheorylong-tailentitiesmusicheritage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Linking names in nineteenth-century music periodicals to a knowledge base is hard because most of the entities are long-tail and about a third have no entry at all (NIL). The paper introduces MHERCL, a gold-annotated benchmark of 875 such sentences, and argues that the bottleneck is not model size but candidate discipline. It offers two systems: ELD, an unsupervised game-theoretic linker, and C-BLINK, which filters an existing retrieval linker's candidates by Wikidata type and date. The paper's strongest claim is that C-BLINK without re-ranking, using a deviation-from-mean score heuristic to decide when no candidate is confident enough, reaches F1 0.76 on MHERCL and 0.67 on HIPE-2020, beating all tested specialized systems and the tested large language models. If that is right, a cheap, open, interpretable linker can outperform far larger models on historical text by refusing uncertain links.

What carries the argument

Two mechanisms carry the argument. The first is the plausibility filter $\Phi$, a set of boolean functions over mention--candidate pairs: $\phi_d$ requires the Wikidata date (birth, inception, publication, or similar property) to precede the document's date, and $\phi_t$ requires the Wikidata "instance of" types to intersect the mention's NER type through a hand-built type taxonomy. C-BLINK applies these by multiplying the bi-encoder similarity score of any implausible candidate by zero before re-ranking. The second mechanism is the NIL rule acting on the surviving score vector $S = [s_0, \dots, s_n]$; the winning rule, deviation from mean, predicts NIL when $(s_0 - \mu(S)) / ((s_0 + \mu(S))/2) < \tau$. ELD uses a different engine: replicator dynamics from evolutionary game theory, where mentions, surrounding content words, and candidate entities are players and strategies, and the payoff matrix is built from mBERT and sense embeddings; iterated updates let the sentence settle on a consistent global labeling.

What would settle it

Count, on MHERCL, how often the gold QID is present in C-BLINK's top-$k$ candidates but is not the top-scoring plausible candidate; the paper's own error analysis shows many such cases become false NIL predictions. If that share is substantial, the assumption that score magnitude tracks correctness fails. A simpler version: rerun the best C-BLINK configuration with predicted NER types instead of manually annotated types and check whether its F1 advantage over BLINK disappears.

Watch

Extended reading notes

Core claim

The central claim is that historical entity linking is best served by making fewer, better-constrained guesses rather than by more expressive models. Concretely, C-BLINK zeroes out the bi-encoder scores of candidate entities whose Wikidata date is not before the document's date ($\phi_d$) or whose Wikidata type is incompatible with the mention's manually annotated NER class ($\phi_t$); after that filter, the deviation-from-mean heuristic answers NIL when the top score is not sufficiently separated from the mean of the candidate scores. In the paper's evaluation this configuration reaches F1 0.76 on MHERCL and 0.67 on HIPE-2020, surpassing every specialized entity linker tested and both tested LLMs (GPT-4 o1-mini at 0.60 on MHERCL and 0.68 on HIPE-2020, and LLAMA 3.3 70B at 0.61 and 0.60). The paper likewise claims that ELD, an unsupervised game-theoretic linker where mentions and context words negotiate via replicator dynamics, avoids popularity bias and reaches F1 0.58 on MHERCL and 0.62 on HIPE-2020. The takeaway is that a simpler retrieval-based neural model enhanced with targeted heuristics can outperform larger, more expensive, closed-source models in long-tail, domain-specific scenarios requiring robust NIL handling.

Load-bearing premise

The load-bearing premise is that after filtering by type and date, the correct entity, when it exists in the knowledge base, will have the highest similarity score; the paper notes that its retriever was not trained to guarantee this, and the type filter is evaluated with manually annotated entity classes rather than classes a real system would predict.

Editorial extensions

If this is right

  • Retrieval-based linking with knowledge-graph constraints should be a strong default for historical text: it is cheaper than calling a large model, open, and higher-scoring in these experiments.
  • NIL handling is decisive: because always predicting NIL already earns F1 0.43 on HIPE-2020, models that cannot decline a link are structurally handicapped on historical benchmarks.
  • Type and date filtering specifically removes anachronistic and wrong-type errors that dense retrieval makes, and it reduces the tendency to choose popular but implausible entities.
  • By observing that ELD's replicator dynamics reinforce popularity bias even when the underlying representations are less biased, the paper identifies score and representation bias, not just retriever architecture, as a target for future work.
  • MHERCL, with its long-tail distribution and 30% NIL share, provides a complementary testbed to HIPE-2020 for evaluating long-tail and NIL-sensitive linking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested pipeline variation that would bound the real-world gain: replace the manually annotated NER classes used by $\phi_t$ with a predicted NER tagger and measure the F1 drop; the reported 0.76 is an upper bound under perfect type information.
  • Because the deviation-from-mean rule only looks at the shape of candidate scores, it should transfer to other dense linkers and to retrieval-augmented generation, where the gap between the best and the mean candidate could gate whether the LLM answers at all.
  • ELD's game-theoretic consensus and C-BLINK's filters are complementary: integrating time and type constraints into ELD's payoff matrix, rather than post-filtering candidates, could raise its 0.58 MHERCL score while preserving its low popularity bias.
  • The 14% OCR-noise slice of MHERCL provides a ready-made test set for OCR post-correction claims, since the paper's own error analysis shows OCR errors are concentrated in wrongly linked mentions across all models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MHERCL, a new gold-standard benchmark for named entity recognition, classification, and linking on historical musical-periodical text, with 875 manually annotated sentences from 76 periodicals (1823–1900), reported Inter-Annotator Agreement of 0.82, and a high share of long-tail and NIL entities. The authors evaluate several off-the-shelf entity linkers and LLMs, finding that they underperform on this benchmark. They then propose two models: ELD, an unsupervised game-theoretic linker, and C-BLINK, an extension of BLINK that filters candidates using time and type constraints from Wikidata and adds NIL prediction through threshold-based or learned heuristics. Experiments on MHERCL and HIPE-2020 report that C-BLINK with the 'deviation from mean' NIL heuristic achieves F1 scores of 0.76 and 0.67, respectively, outperforming GPT-4o-mini and LLAMA-3.3 70B.

Significance. If the results are taken at face value, the paper makes two contributions: a publicly released, carefully annotated benchmark for a neglected domain (historical music periodicals), and evidence that simple KG-based constraints plus score-based NIL prediction can beat much larger closed models on historical entity linking. The dataset construction is a genuine asset: the annotation guidelines are detailed, the IAA of 0.82 is respectable, and the popularity analysis convincingly shows that MHERCL is skewed toward long-tail entities. The code and data are publicly available, which supports reproducibility. However, the headline claim about outperforming LLMs rests on an evaluation protocol whose threshold selection is performed on the test set, so the quantitative edge reported in Table 7 is not yet established as a property of a fixed model.

major comments (3)
  1. [Section 5.3 / Table 7] This is a load-bearing issue because the central claim of outperforming LLMs depends on the NIL decision being made in a way that generalizes beyond the test set; as written, the comparison is not a fair evaluation of a defined method.
  2. [Section 5.2, Section 5.3] Additionally, the ML-based NIL classifiers (SVM, logistic regression, decision tree) are stated in Section 5.3 to be trained on HIPE-2020 and evaluated on MHERCL, yet Table 7 also reports HIPE-2020 scores for these classifiers. If those HIPE-2020 numbers are obtained by training and testing on the same set, they are circular and should be removed or replaced with cross-validated or separately trained results.
  3. [Section 4.2 / Section 5.4] The paper's NIL heuristics rely on the assumption that, when the correct entity is in the candidate pool, it will have the highest similarity score (Section 4.2). Section 5.4 concedes that the retriever and re-ranker are not explicitly trained to produce calibrated scores, and Table 14 shows that the most frequent C-BLINK error is precisely predicting NIL while the correct QID is among the candidates. This means the score-magnitude assumption is internally acknowledged to be unreliable. To support the claim that score-based NIL heuristics are robust, the authors should provide a distribution of score gaps for correct versus incorrect links, or demonstrate that the chosen heuristic generalizes across documents or datasets rather than only on the test set where the threshold was tuned.
minor comments (5)
  1. [Section 4.1.1] There is a typo: 'MERCHL' should be 'MHERCL' in the sentence about the context of ELD.
  2. [Section 5.4] The text says 'Table 5.4' when referring to results; it should reference Table 7.
  3. [Section 5.4] The phrase 'favour NIL predcitions' contains a typo ('predcitions' should be 'predictions').
  4. [Section 2.2] The model name is written as 'ReFiNeD' in one place and 'ReFinED' elsewhere; please unify the spelling throughout.
  5. [References] The reference list contains a duplicate entry for Lacerra et al. (2021); the in-text citation 'Lacerra et al., 2021' appears twice with different reference strings and should be consolidated.

Circularity Check

2 steps flagged · score 6.0 of 10

C-BLINK†'s best NIL numbers are selected by sweeping 1,001 thresholds on the test sets, and the HIPE-trained NIL classifiers are also scored in-sample on HIPE-2020.

  1. fitted input called prediction [Section 5.3 (NIL heuristics), Table 7, Section 5.4 (Results)]
    "In our experiments, we extensively test all NIL heuristics listed in Table 6 to assess their impact on the output. For threshold-based heuristics, we experiment with values τ ∈ [0, 1] with increments of 0 .001."

    The NIL decision is made by comparing a score-dispersion statistic to a threshold τ (Table 6), so τ is the parameter that determines whether a mention is labelled NIL or linked. The paper does not describe any validation split or threshold-selection rule: it sweeps τ over 1,001 values and then reports the F1 achieved at the selected τ on the same test sets. The Table 7 row 'C-BLINK† {ϕd, ϕt} Deviation from mean' (τ=0.022, HIPE-2020 0.67, MHERCL 0.76) is therefore a post-selection maximum, not the performance of a NIL policy fixed before seeing test labels. The Section 5.4 claim that C-BLINK† 'can outperform larger, more expensive, and closed-source models' rests on those NIL labels being partially fitted to the evaluation data.

  2. fitted input called prediction [Section 5.3 and Table 7 (SVM, Logistic Regression, Decision Tree rows)]
    "For machine learning approaches, we train classifiers on HIPE-2020 and evaluate their performance on MHERCL."

    Table 7 nonetheless reports HIPE-2020 F1 scores for the SVM, logistic regression, and decision tree NIL classifiers (e.g., C-BLINK† SVM 0.66, logistic regression 0.65, decision tree 0.76). If the classifiers are trained on the HIPE-2020 English test set, as Section 2.1 describes that dataset, then the HIPE-2020 column is in-sample and those numbers are fitted rather than predicted. If a separate HIPE-2020 split is intended, the paper does not say so. Either way, the stated protocol does not establish that these NIL classifiers generalise to the data on which their HIPE-2020 scores are reported.

full rationale

The derivation chain for C-BLINK is mostly transparent: BLINK's candidate scores are filtered by external Wikidata facts, and the NIL heuristics are defined in Table 6 rather than derived from the target result. ELD is an unsupervised game-theoretic adaptation of published relaxation-labeling work and does not depend on a fitted NIL threshold. The dataset contribution and the popularity analyses are independent of the threshold-fitting issue. The circularity is concentrated in the headline NIL result: Section 5.3 sweeps τ over 1,001 values, and Table 7 reports the F1 at the selected τ on the same test sets, with no validation split or threshold-selection procedure described. The Section 5.4 comparison against LLMs therefore reports post-selection maxima rather than a fixed-policy prediction. A second, smaller instance of fitted input being reported as prediction is the HIPE-trained NIL classifiers being scored on HIPE-2020 in Table 7. The use of gold NER types and the hand-built type taxonomy is an oracle-feature issue rather than a circular derivation, and the self-citations (Tripodi et al. 2022; Tripodi and Navigli 2019) are not load-bearing for the headline claim. Overall, the central 'can outperform' claim partially reduces to test-set threshold fitting, so the score is 6 rather than 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on several hand-tuned components: NIL thresholds selected per test set, a manually built type taxonomy, and a hand-ranked list of Wikidata time properties. The game-theoretic and top-score assumptions are stated but not independently validated.

free parameters (3)
  • NIL heuristic thresholds (tau) = e.g., fixed threshold 0.443 for C-BLINK on MHERCL and 0.626 on HIPE-2020; deviation-from-top 0.003/0.011…
    Selected by grid search over [0,1] in steps of 0.001 on each test set (Section 5.3, Table 7), so they are fitted to the evaluation data.
  • Type taxonomy mapping = ad-hoc taxonomy in Table C3
    Manually constructed by the annotators to map Wikidata P31 types to MHERCL/HIPE-2020 NER classes (Section 5.2); used in the type plausibility filter.
  • Time-property retrieval order = 15 Wikidata properties ranked as in Table D5
    Chosen by qualitative analysis of property names and descriptions (Section 5.1), affecting which date is used for temporal filtering.
assumptions (4)
  • standard math Replicator dynamics converge to a consistent labeling for the EL game
    Inherited from relaxation labelling theory (Hummel and Zucker 1983; Miller and Zucker 1991), used in Section 4.1.1 without new proof in the EL context.
  • domain assumption The correct entity, if present in the KB, has the highest similarity among plausible candidates
    Stated in Section 4.2: 'we assume that if the correct entity is present in the KB, then it must be the one with the highest similarity with respect to the document's mention.' This underpins all NIL heuristics.
  • domain assumption Wikidata type and time information is sufficiently complete and accurate for historical entities
    The plausibility filters (Sections 5.1 and 5.2) rely on P31 and time properties; missing or wrong Wikidata values would remove correct candidates.
  • ad hoc to paper mBERT entity embeddings and Ares sense embeddings live in a compatible latent space
    ELD (Section 4.1.2) computes pairwise similarities between mention, sense, and entity embeddings from different sources without a demonstrated alignment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Musical Heritage Historical Entity Linking." pith.science (2026). https://pith.science/paper/7D5KFXUA

@misc{pith2026250209168,
  author       = {Pith},
  title        = {Pith review of: Musical Heritage Historical Entity Linking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7D5KFXUA}},
  note         = {Machine review of arXiv:2502.09168}
}
read the original abstract

Linking named entities occurring in text to their corresponding entity in a Knowledge Base (KB) is challenging, especially when dealing with historical texts. In this work, we introduce Musical Heritage named Entities Recognition, Classification and Linking (MHERCL), a novel benchmark consisting of manually annotated sentences extrapolated from historical periodicals of the music domain. MHERCL contains named entities under-represented or absent in the most famous KBs. We experiment with several State-of-the-Art models on the Entity Linking (EL) task and show that MHERCL is a challenging dataset for all of them. We propose a novel unsupervised EL model and a method to extend supervised entity linkers by using Knowledge Graphs (KGs) to tackle the main difficulties posed by historical documents. Our experiments reveal that relying on unsupervised techniques and improving models with logical constraints based on KGs and heuristics to predict NIL entities (entities not represented in the KB of reference) results in better EL performance on historical documents.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 48 canonical work pages

  1. [1]

    , Str \"o tgen , J

    bchapter Agarwal , P. , Str \"o tgen , J. , Corro , L. , Hoffart , J. , Weikum , G. : dia NED : Time-aware named entity disambiguation for diachronic corpora . In: Gurevych , I. , Miyao , Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pp. 686 -- 693 . Association for Computation...

    work page doi:10.18653/v1/p18-2109 2018

  2. [2]

    , Guo , Z

    barticle Alani , H. , Guo , Z. , Barbosa , D. : Robust Named Entity Disambiguation with Random Walks . Semantic Web Journal 9 ( 4 ), 459 -- 479 ( 2018 ) 10.3233/SW-170273 barticle

    doi:10.3233/sw-170273 2018

  3. [3]

    , Fisher , J

    bchapter Ayoola , T. , Fisher , J. , Pierleoni , A. : Improving entity disambiguation by reasoning over a knowledge base . In: Carpuat , M. , Marneffe , M.-C. , Meza Ruiz , I.V. (eds.) Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pp. 2899 -- 2912 . Associat...

    work page doi:10.18653/v1/2022.naacl-main.210 2022

  4. [4]

    , Bonial , C

    bchapter Banarescu , L. , Bonial , C. , Cai , S. , Georgescu , M. , Griffitt , K. , Hermjakob , U. , Knight , K. , Koehn , P. , Palmer , M. , Schneider , N. : A bstract M eaning R epresentation for sembanking . In: Pareja-Lora , A. , Liakata , M. , Dipper , S. (eds.) Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse...

    work page 2013

  5. [5]

    , Procopio , L

    bchapter Barba , E. , Procopio , L. , Navigli , R. : E xt E n D : Extractive entity disambiguation . In: Muresan , S. , Nakov , P. , Villavicencio , A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 2478 -- 2488 . Association for Computational Linguistics , Dublin, Ireland ( 202...

    work page doi:10.18653/v1/2022.acl-long.177 2022

  6. [6]

    , Skapars , A

    bchapter Benkhedda , Y. , Skapars , A. , Schlegel , V. , Nenadic , G. , Batista-Navarro , R. : Enriching the metadata of community-generated digital content through entity linking: An evaluative comparison of state-of-the-art models . In: Bizzoni , Y. , Degaetano-Ortlieb , S. , Kazantseva , A. , Szpakowicz , S. (eds.) Proceedings of the 8th Joint SIGHUM W...

    work page 2024

  7. [7]

    , Armand , C

    bchapter Blouin , B. , Armand , C. , Henriot , C. : A dataset for named entity recognition and entity linking in C hinese historical newspapers . In: Calzolari , N. , Kan , M.-Y. , Hoste , V. , Lenci , A. , Sakti , S. , Xue , N. (eds.) Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (L...

    work page 2024

  8. [8]

    , González-Gallardo , C.-E

    bchapter Boros , E. , González-Gallardo , C.-E. , Giamphy , E. , Hamdi , A. , 0001 , J.G.M. , Doucet , A. : Knowledge-based contexts for historical named entity recognition and linking . In: Faggioli , G. , Ferro , N. , Hanbury , A. , Potthast , M. (eds.) Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum . CEUR Wo...

    work page 2022

Show all 73 references
  1. [9]

    , Pontes , E.L

    bchapter Boros , E. , Pontes , E.L. , Cabrera-Diego , L.A. , Hamdi , A. , Moreno , J.G. , Sid \`e re , N. , Doucet , A. : Robust named entity recognition and linking on historical multilingual documents . In: Working Notes of CLEF 2020 , vol. 2696 , pp. 1 -- 17 ( 2020 ). https...

    2020

  2. [10]

    , Pa s ca , M

    bchapter Bunescu , R. , Pa s ca , M. : Using encyclopedic knowledge for named entity disambiguation . In: McCarthy , D. , Wintner , S. (eds.) 11th Conference of the E uropean Chapter of the Association for Computational Linguistics , pp. 9 -- 16 . Association for Computational...

    2006

  3. [11]

    : Fast K rippendorff: Fast computation of K rippendorff's alpha agreement measure

    botherref Castro , S. : Fast K rippendorff: Fast computation of K rippendorff's alpha agreement measure. GitHub. Available at: https://github.com/pln-fing-udelar/fast-krippendorff. Accessed 13 Aug 2024 (2017) botherref

    2017

  4. [12]

    , Gudipati , P

    bchapter Chen , A. , Gudipati , P. , Longpre , S. , Ling , X. , Singh , S. : Evaluating entity disambiguation and the role of popularity in retrieval-based NLP . In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internatio...

    2021 doi

  5. [13]

    , Saha Roy , R

    bchapter Christmann , P. , Saha Roy , R. , Weikum , G. : Beyond ned: Fast and effective search space reduction for complex question answering over knowledge bases . In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining . WSDM '22 , pp. 172...

    2022

  6. [14]

    , Izacard , G

    bchapter De Cao , N. , Izacard , G. , Riedel , S. , Petroni , F. : Autoregressive entity retrieval . In: 9th International Conference on Learning Representations . OpenReview.net , Online, Austria ( 2021 ). https://openreview.net/forum?id=5k8F6UU39V bchapter

    2021

  7. [15]

    , Wu , L

    barticle De Cao , N. , Wu , L. , Popat , K. , Artetxe , M. , Goyal , N. , Plekhanov , M. , Zettlemoyer , L. , Cancedda , N. , Riedel , S. , Petroni , F. : Multilingual autoregressive entity linking . Transactions of the Association for Computational Linguistics 10 , 274 -- 290...

    2022 doi

  8. [16]

    , Chang , M.-W

    bchapter Devlin , J. , Chang , M.-W. , Lee , K. , Toutanova , K. : BERT : Pre-training of deep bidirectional transformers for language understanding . In: Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human ...

    2019 doi

  9. [17]

    , Chen , J

    bchapter Dong , H. , Chen , J. , He , Y. , Liu , Y. , Horrocks , I. : Reveal the Unknown: Out-of-Knowledge-Base Mention Discovery with Entity Linking . In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management . CIKM '23 , pp. 452 -- 462 ...

    2023

  10. [18]

    , Colavizza , G

    bchapter Ehrmann , M. , Colavizza , G. , Rochat , Y. , Kaplan , F. : Diachronic Evaluation of NER Systems on Old Newspapers . In: Dipper , S. , Neubarth , F. , Zinsmeister , H. (eds.) Proceedings of the 13th Conference on Natural Language Processing (KONVENS 2016) , pp. 97 -- ...

    2016

  11. [19]

    , Hamdi , A

    botherref Ehrmann , M. , Hamdi , A. , Pontes , E.L. , Romanello , M. , Doucet , A. : Named entity recognition and classification in historical documents: A survey. ACM Computing Surveys 56(2) (2023) 10.1145/3604931 botherref

    2023 doi

  12. [20]

    , Romanello , M

    bchapter Ehrmann , M. , Romanello , M. , Doucet , A. , Clematide , S. : Introducing the HIPE 2022 Shared Task : Named Entity Recognition and Linking in Multilingual Historical Documents . In: Hagen , M. , Verberne , S. , Macdonald , C. , Seifert , C. , Balog , K. , N rv g , K....

    2022 doi

  13. [21]

    , Romanello , M

    bchapter Ehrmann , M. , Romanello , M. , Fluckiger , A. , Clematide , S. : Extended Overview of CLEF HIPE 2020: Named Entity Processing on Historical Newspapers . In: Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum , vol. 2696 . Thessaloniki, Greece , ...

    2020 doi

  14. [22]

    , Romanello , M

    bchapter Ehrmann , M. , Romanello , M. , Fl\" u ckiger , A. , Clematide , S. : Overview of CLEF HIPE 2020: Named Entity Recognition and Linking on Historical Newspapers . In: Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conference o...

    2020 doi

  15. [23]

    , Romanello , M

    bchapter Ehrmann , M. , Romanello , M. , Najem-Meyer , S. , Doucet , A. , Clematide , S. : Extended overview of hipe-2022: Named entity recognition and linking in multilingual historical documents . In: Faggioli , G. , Ferro , N. , Hanbury , A. , Potthast , M. (eds.) Working N...

    2022 doi

  16. [24]

    , Ringgaard , M

    botherref Gabrilovich , E. , Ringgaard , M. , Subramanya , A. : FACC1: Freebase annotation of ClueWeb corpora, Version 1 (Release date 2013-06-26, Format version 1, Correction level 0) . Web Download. Available at http://lemurproject.org/clueweb09/ and http://lemurproject.org/...

    2013

  17. [25]

    : The AQUAINT Corpus of English News Text

    botherref Graff , D. : The AQUAINT Corpus of English News Text. Linguistic Data Consortium (2002). https://catalog.ldc.upenn.edu/LDC2002T31 botherref

    2002

  18. [26]

    , Dong , L

    bchapter Gu , Y. , Dong , L. , Wei , F. , Huang , M. : Minillm: Knowledge distillation of large language models . In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net , ??? ( 2024 ). https://openrevie...

    2024

  19. [27]

    , Garcia-Dominguez , A

    barticle Guellil , I. , Garcia-Dominguez , A. , Lewis , P.R. , Hussain , S. , Smith , G. : Entity linking for English and other languages: A survey . Knowledge and Information Systems 66 ( 7 ), 3773 -- 3824 ( 2024 ) 10.1007/s10115-023-02059-2 barticle

    2024 doi

  20. [28]

    , Linhares Pontes , E

    bchapter Hamdi , A. , Linhares Pontes , E. , Boros , E. , Nguyen , T.T.H. , Hackl , G. , Moreno , J.G. , Doucet , A. : A Multilingual Dataset for Named Entity Recognition, Entity Linking and Stance Detection in Historical Newspapers . In: Proceedings of the 44th International ...

    2021

  21. [29]

    , Krippendorff , K

    barticle Hayes , A.F. , Krippendorff , K. : Answering the call for a standard reliability measure for coding data . Communication Methods and Measures 1 ( 1 ), 77 -- 89 ( 2007 ) 10.1080/19312450709336664 barticle

    2007 doi

  22. [30]

    , Seufert , S

    bchapter Hoffart , J. , Seufert , S. , Nguyen , D.B. , Theobald , M. , Weikum , G. : KORE: keyphrase overlap relatedness for entity disambiguation . In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management . CIKM '12 , pp. 545 -- 554 . A...

    2012

  23. [31]

    , Yosef , M.A

    bchapter Hoffart , J. , Yosef , M.A. , Bordino , I. , F \"u rstenau , H. , Pinkal , M. , Spaniol , M. , Taneva , B. , Thater , S. , Weikum , G. : Robust disambiguation of named entities in text . In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language P...

    2011

  24. [32]

    , Zucker , S.W

    barticle Hummel , R.A. , Zucker , S.W. : On the Foundations of Relaxation Labeling Processes . IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-5 ( 3 ), 267 -- 287 ( 1983 ) 10.1109/TPAMI.1983.4767390 barticle

    1983

  25. [33]

    , Vossen , P

    bchapter Ilievski , F. , Vossen , P. , Schlobach , S. : Systematic study of long tail phenomena in entity linking . In: Bender , E.M. , Derczynski , L. , Isabelle , P. (eds.) Proceedings of the 27th International Conference on Computational Linguistics , pp. 664 -- 674 . Assoc...

    2018

  26. [34]

    , Caciularu , A

    bchapter Jacovi , A. , Caciularu , A. , Goldman , O. , Goldberg , Y. : Stop uploading test data in plain text: Practical strategies for mitigating data contamination by evaluation benchmarks . In: Bouamor , H. , Pino , J. , Bali , K. (eds.) Proceedings of the 2023 Conference o...

    2023 doi

  27. [35]

    , Grishman , R

    bchapter Ji , H. , Grishman , R. , Dang , H.T. , Griffit , K. , Ellis , J. : Overview of the tac 2010 knowledge base population track . In: Proceedings of the 2010 Text Analysis Conference ( 2010 ) bchapter

    2010

  28. [36]

    , Martin , J.H

    bbook Jurafsky , D. , Martin , J.H. : Speech and Language Processing , ( 2023 ). Draft of January 7, 2023. Available at: https://web.stanford.edu/ jurafsky/slp3/. Accessed 13 Aug 2024 bbook

    2023

  29. [37]

    , Deng , H

    bchapter Kandpal , N. , Deng , H. , Roberts , A. , Wallace , E. , Raffel , C. : Large language models struggle to learn long-tail knowledge . In: Proceedings of the 40th International Conference on Machine Learning . ICML'23 . JMLR.org , Honolulu, Hawaii, USA ( 2023 ) bchapter

    2023

  30. [38]

    , Ganea , O.-E

    bchapter Kolitsas , N. , Ganea , O.-E. , Hofmann , T. : End-to-end neural entity linking . In: Korhonen , A. , Titov , I. (eds.) Proceedings of the 22nd Conference on Computational Natural Language Learning , pp. 519 -- 529 . Association for Computational Linguistics , Brussel...

    2018 doi

  31. [39]

    , Pasini , T

    bchapter Lacerra , C. , Pasini , T. , Tripodi , R. , Navigli , R. : Alasca: an automated approach for large-scale lexical substitution . In: Zhou , Z. (ed.) Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Mont...

    2021 doi

  32. [40]

    , Tripodi , R

    bchapter Lacerra , C. , Tripodi , R. , Navigli , R. : G ene S is: A G enerative A pproach to S ubstitutes in C ontext . In: Moens , M.-F. , Huang , X. , Specia , L. , Yih , S.W.-t. (eds.) Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , ...

    2021 doi

  33. [41]

    , Flanigan , J

    barticle Li , C. , Flanigan , J. : Task contamination: Language models may not be few-shot anymore . Proceedings of the AAAI Conference on Artificial Intelligence 38 ( 16 ), 18471 -- 18480 ( 2024 ) 10.1609/aaai.v38i16.29808 barticle

    2024 doi

  34. [42]

    , Asai , A

    bchapter Mallen , A. , Asai , A. , Zhong , V. , Das , R. , Khashabi , D. , Hajishirzi , H. : When not to trust language models: Investigating effectiveness of parametric and non-parametric memories . In: Rogers , A. , Boyd-Graber , J. , Okazaki , N. (eds.) Proceedings of the 6...

    2023 doi

  35. [43]

    , Schnaitter , H

    bbook Menzel , S. , Schnaitter , H. , Zinck , J. , Petras , V. , Neudecker , C. , Labusch , K. , Leitner , E. , Rehm , G. : In: Franke-Maier , M. , Kasprzik , A. , Ledl , A. , Schürmann , H. (eds.) Named Entity Linking mit Wikidata und GND – Das Potenzial handkuratierter und s...

    2021 doi

  36. [44]

    , Zucker , S.W

    barticle Miller , D.A. , Zucker , S.W. : Copositive-plus lemke algorithm solves polymatrix games . Operations Research Letters 10 ( 5 ), 285 -- 290 ( 1991 ) 10.1016/0167-6377(91)90015-H barticle

    1991 doi

  37. [45]

    : WordNet: A Lexical Database for English

    barticle Miller , G.A. : WordNet: A Lexical Database for English . Communications of the ACM 38 ( 11 ), 39 -- 41 ( 1995 ) 10.1145/219717.219748 barticle

    1995

  38. [46]

    , Strassel , S

    botherref Mitchell , A. , Strassel , S. , Huang , S. , Zakhary , R. : ACE 2004 Multilingual Training Corpus LDC2005T09. Linguistic Data Consortium, Philadelphia (2005) botherref

    2005

  39. [47]

    , Lehmann , J

    barticle M \"o ller , C. , Lehmann , J. , Usbeck , R. : Survey on English Entity Linking on Wikidata : Datasets and approaches . Semantic Web 13 ( 6 ), 925 -- 966 ( 2022 ) 10.3233/SW-212865 barticle

    2022 doi

  40. [48]

    : Non-cooperative games

    botherref Nash , J. : Non-cooperative games. Annals of mathematics, 286--295 (1951) botherref

    1951

  41. [49]

    , Huguet Cabot , P.-L

    bchapter Orlando , R. , Huguet Cabot , P.-L. , Barba , E. , Navigli , R. : R e L i K : Retrieve and L in K , fast and accurate entity linking and relation extraction on an academic budget . In: Ku , L.-W. , Martins , A. , Srikumar , V. (eds.) Findings of the Association for Co...

    2024

  42. [50]

    : The Dynamics of Nonlinear Relaxation Labeling Processes

    barticle Pelillo , M. : The Dynamics of Nonlinear Relaxation Labeling Processes . J. Math. Imaging Vis. 7 ( 4 ), 309 -- 323 ( 1997 ) 10.1023/A:1008255111261 barticle

    1997 doi

  43. [51]

    , Piktus , A

    bchapter Petroni , F. , Piktus , A. , Fan , A. , Lewis , P. , Yazdani , M. , De Cao , N. , Thorne , J. , Jernite , Y. , Karpukhin , V. , Maillard , J. , Plachouras , V. , Rockt \"a schel , T. , Riedel , S. : KILT : a benchmark for knowledge intensive language tasks . In: Proce...

    2021

  44. [52]

    , Kassner , N

    botherref Plekhanov , M. , Kassner , N. , Popat , K. , Martin , L. , Merello , S. , Kozlovskii , B. , Dreyer , F.A. , Cancedda , N. : Multilingual end to end entity linking. arXiv preprint arXiv:2306.08896 (2023) botherref

    2023 arXiv

  45. [53]

    , Cabrera-Diego , L.A

    barticle Pontes , E.L. , Cabrera-Diego , L.A. , Moreno , J.G. , Boros , E. , Hamdi , A. , Doucet , A. , Sid \`e re , N. , Coustaty , M. : Melhissa: a multilingual entity linking architecture for historical press articles . International Journal on Digital Libraries 23 , 133 --...

    2022

  46. [54]

    , Bhargav , S

    bchapter Provatorova , V. , Bhargav , S. , Vakulenko , S. , Kanoulas , E. : Robustness Evaluation of Entity Disambiguation Using Prior Probes: the Case of Entity Overshadowing . In: Moens , M.-F. , Huang , X. , Specia , L. , Yih , S.W.-t. (eds.) Proceedings of the 2021 Confere...

    2021 doi

  47. [55]

    , Xamena , E

    bchapter Ramirez - Orta , J.A. , Xamena , E. , Maguitman , A.G. , Milios , E.E. , Soto , A.J. : Post-ocr document correction with large ensembles of character sequence-to-sequence models . In: Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Co...

    2022

  48. [56]

    , Pelillo , M

    barticle Rota Bulò , S. , Pelillo , M. , Bomze , I.M. : Graph-based quadratic optimization: A fast evolutionary approach . Computer Vision and Image Understanding 115 ( 7 ), 984 -- 995 ( 2011 ) 10.1016/j.cviu.2010.12.004 . Special issue on Graph-Based Representations in Comput...

    2011 doi

  49. [57]

    , Campos , J

    bchapter Sainz , O. , Campos , J. , Garc \' a-Ferrero , I. , Etxaniz , J. , Lacalle , O.L. , Agirre , E. : NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark . In: Bouamor , H. , Pino , J. , Bali , K. (eds.) Findings of the Association ...

    2023 doi

  50. [58]

    , Tan , M.A

    bchapter Santini , C. , Tan , M.A. , Bruns , O. , Tietz , T. , Posthumus , E. , Sack , H. : Knowledge extraction for art history: the case of vasari's the lives of the artists . In: Paschke , A. , Rehm , G. , Neudecker , C. , Pintscher , L. (eds.) Proceedings of the Third Conf...

    2022

  51. [59]

    , Pasini , T

    bchapter Scarlini , B. , Pasini , T. , Navigli , R. : With more contexts comes better performance: Contextualized sense embeddings for all-round word sense disambiguation . In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp....

    2020 doi

  52. [60]

    , Shelmanov , A

    barticle Sevgili , \"O . , Shelmanov , A. , Arkhipov , M. , Panchenko , A. , Biemann , C. : Neural Entity Linking: A Survey of Models Based on Deep Learning . Semantic Web 13 ( 3 ), 527 -- 570 ( 2022 ) 10.3233/SW-222986 barticle

    2022 doi

  53. [61]

    , Xu , Y.E

    botherref Sun , K. , Xu , Y.E. , Zha , H. , Liu , Y. , Dong , X.L. : Head-to- Tail : How Knowledgeable Are Large Language Models ( LLMs )? A . K . A . Will LLMs Replace Knowledge Graphs ? arXiv (2024) botherref

    2024

  54. [62]

    , Jonker , L.B

    barticle Taylor , P.D. , Jonker , L.B. : Evolutionary stable strategies and game dynamics . Mathematical Biosciences 40 ( 1 ), 145 -- 156 ( 1978 ) 10.1016/0025-5564(78)90077-9 barticle

    1978 doi

  55. [63]

    , Conia , S

    bchapter Tedeschi , S. , Conia , S. , Cecconi , F. , Navigli , R. : N amed E ntity R ecognition for E ntity L inking: W hat works and what ' s next . In: Moens , M.-F. , Huang , X. , Specia , L. , Yih , S.W.-t. (eds.) Findings of the Association for Computational Linguistics: ...

    2021 doi

  56. [64]

    , Blloshmi , R

    bchapter Tripodi , R. , Blloshmi , R. , Levis Sullam , S. : Evaluating multilingual sentence representation models in a real case scenario . In: Proceedings of the Thirteenth Language Resources and Evaluation Conference , pp. 2928 -- 2939 . European Language Resources Associat...

    2022

  57. [65]

    , Navigli , R

    bchapter Tripodi , R. , Navigli , R. : Game theory meets embeddings: a unified framework for word sense disambiguation . In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pr...

    2019 doi

  58. [66]

    , Hasibi , F

    bchapter Hulst , J.M. , Hasibi , F. , Dercksen , K. , Balog , K. , Vries , A.P. : Rel: An entity linker standing on the shoulders of giants . In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . SIGIR '20 , pp. 2...

    2020

  59. [67]

    , Dong , Y

    barticle Wang , J. , Dong , Y. : Measurement of text similarity: A survey . Inf. 11 ( 9 ), 421 ( 2020 ) 10.3390/INFO11090421 barticle

    2020 doi

  60. [68]

    : Evolutionary Game Theory

    bbook Weibull , J.W. : Evolutionary Game Theory . MIT press , Cambridge, MA, USA ( 1997 ) bbook

    1997

  61. [69]

    , Petroni , F

    bchapter Wu , L. , Petroni , F. , Josifoski , M. , Riedel , S. , Zettlemoyer , L. : Scalable Zero-shot Entity Linking with Dense Entity Retrieval . In: Webber , B. , Cohn , T. , He , Y. , Liu , Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Langua...

    2020 doi

  62. [70]

    , Kaffee , L.-A

    bchapter Zaporojets , K. , Kaffee , L.-A. , Deleu , J. , Demeester , T. , Develder , C. , Augenstein , I. : Temp EL : Linking dynamically evolving and newly emerging entities . In: Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (...

    2022

  63. [71]

    , Hua , W

    bchapter Zhang , W. , Hua , W. , Stratos , K. : Ent QA : Entity linking as question answering . In: International Conference on Learning Representations ( 2022 ). https://openreview.net/forum?id=US2rTP5nm\_ bchapter

    2022

  64. [72]

    , Yu , J

    bchapter Zhu , F. , Yu , J. , Jin , H. , Hou , L. , Li , J. , Sui , Z. : Learn to not link: Exploring NIL prediction in entity linking . In: Rogers , A. , Boyd-Graber , J. , Okazaki , N. (eds.) Findings of the Association for Computational Linguistics: ACL 2023 , pp. 10846 -- ...

    2023 doi

  65. [73]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.