Pith. sign in

REVIEW 2 major objections 4 minor 51 references

Dynamic Injection of Entity Knowledge into Dense Retrievers

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A small attention layer that looks up Wikipedia entities lifts dense-retriever top-20 accuracy by 12.6% on entity-heavy questions.

desk verdict Useful entity-aware dense retriever, but a time-mismatched entity knowledge base leaves the headline EQ gain unproven until a matched-dump control is run. read the letter →

arxiv 2507.03922 v2 pith:H3EQ2AEE submitted 2025-07-05 cs.CL

Ikuya Yamada , Ryokan Ri , Takeshi Kojima , Yusuke Iwasawa , Yutaka Matsuo This is my paper
classification cs.CL
keywords entityknowledgeinjectiondenseretrievalopen-domainquestionansweringembeddingslinkingBERTlong-tailentitiesretrieval-augmentedgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to show that the failure of dense retrievers on less-frequent entities is not a fixed limitation of their learned encoders but can be remedied by giving the encoder access to external entity knowledge at inference time. It proposes the Knowledgeable Passage Retriever (KPR), which adds a context-entity attention layer on top of a BERT-based retriever and supplies it with frozen entity embeddings built from Wikipedia. The result is a 12.6% top-20 accuracy gain on EntityQuestions over the vanilla DPR model, consistent improvements on Natural Questions and TriviaQA, and state-of-the-art performance among similarly sized models when KPR is built on the off-the-shelf bge-base retriever. The practical point is that entity knowledge can be added or updated without retraining the retriever, which matters for keeping retrieval systems current.

What carries the argument

The central mechanism is the context-entity attention layer: a single-head attention layer placed above BERT's [CLS] output. The [CLS] embedding is the query; a set of entity embeddings, each formed by BERT inference over Wikipedia passages mentioning the entity, serve as both keys and values after passing through learned projections and being added to learned position embeddings, and a no-op embedding keeps the sequence non-empty. The activation is a sigmoid with a length bias, so each entity's relevance is scored independently rather than competing through softmax. Because the entity embeddings are frozen and the linker is a dictionary, the whole entity side of the model can be swapped at inference time, which is what makes the knowledge dynamic.

What would settle it

Rebuild KPR's entity embeddings and dictionary linker from the same December 2018 Wikipedia dump used for the retrieval corpus and rerun the EntityQuestions comparison; if the 12.6% gain over DPR largely disappears, the reported benefit comes from temporal leakage rather than from the attention layer's ability to use entity knowledge.

Watch

Extended reading notes

Core claim

The paper claims that injecting entity knowledge through a dedicated attention mechanism is enough to make a dense retriever handle long-tail entity queries. KPR takes DPR's [CLS] embedding as the query, looks up entity candidates mentioned in the input via a dictionary linker, and runs one attention head in which those entities' frozen embeddings serve as keys and values. Since the entity embeddings are computed outside the base model and kept fixed during training, the retriever can incorporate new entity knowledge after training simply by recomputing those embeddings and updating the linker dictionary. The authors support the claim by showing that KPR beats the same architecture without the layer, beats an earlier entity-token-injection method when both use the same linker, and remains competitive with BM25 on rare entities while dominating it on frequent ones.

Load-bearing premise

The load-bearing assumption is that the entity knowledge fed in is not secretly newer than the passages being searched: the entity embeddings and linker come from an April 2024 Wikipedia dump while the corpus is December 2018, and the paper does not address this mismatch.

Editorial extensions

If this is right

  • Top-20 accuracy on EntityQuestions rises from 56.8 to 69.4 when the same BERT-based DPR is wrapped in KPR, a 12.6-point gain concentrated on queries with less-frequent entities.
  • KPR built on bge-base reaches 76.8 on EntityQuestions and 81.5 on TriviaQA, the highest top-20 accuracies among similarly sized retrievers compared in the paper.
  • New entity knowledge can be added after training by recomputing the frozen entity embeddings and updating the dictionary linker, with no retraining of the base retriever.
  • Even a noisy, no-disambiguation dictionary linker outperforms a high-precision entity linker, because the attention layer learns to downweight irrelevant candidate entities.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported gains survive rebuilding the entity embeddings and linker from the December 2018 Wikipedia dump used for the retrieval corpus, KPR is a genuine architectural fix for long-tail entity retrieval rather than a beneficiary of temporal leakage.
  • The attention weights' implicit preference for correct entities suggests the same layer could be reused as a lightweight query entity linker or as a reranker that rewards passages containing the attended entity.
  • A dynamic-update scenario the paper does not test—querying an old passage corpus with entity knowledge about recent events—would reveal whether the frozen-but-updatable embeddings handle knowledge drift or simply inject newer facts retroactively.
  • The reported 0.18% FLOP overhead assumes the entity embeddings can be served from CPU memory or disk, so a practical next step is measuring wall-clock latency under that setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces the Knowledgeable Passage Retriever (KPR), an extension of BERT-style dense retrievers that adds a context-entity attention layer over the [CLS] embedding. Entity mentions are detected either by a dictionary linker or by ReFinED, represented by BERT-computed entity embeddings that are averaged from masked Wikipedia passages and frozen during training, and attended to with a sigmoid attention scaled by entity length. KPR is evaluated on EntityQuestions, Natural Questions, TriviaQA, and MS MARCO, reporting a 12.6-point top-20 accuracy gain over DPR on EntityQuestions, consistent improvements when built on RetroMAE, and state-of-the-art base-size results when built on bge-base. The paper also provides ablations over embedding sources, activation functions, and a frequency-binned analysis of EntityQuestions.

Significance. If the reported gains survive a matched-dump control, KPR is a simple and modular contribution to entity-aware dense retrieval: it requires no retraining of the base encoder to exploit updated entity knowledge, and it ships with code, models, multi-seed confidence intervals, and careful ablations. The central empirical risk is the temporal mismatch between the April 2024 entity resource and the December 2018 passage corpus; the significance of the headline results therefore depends on the outcome of the additional experiment requested below. The paper does not exhibit circularity or test-set fitting: entity embeddings are frozen and precomputed, and hyperparameters are selected on the NQ development set.

major comments (2)
  1. [Appendix C, Section 2, Section 3.1] The entity embeddings and the dictionary linker are built from the April 2024 Wikipedia dump (7.2M entities, Appendix C), while the retrieval corpus is the December 2018 dump used by Karpukhin et al. (2020). This five-year gap is not discussed in the paper, and it is a direct confound for the headline 12.6-point EntityQuestions gain (Table 1) and for the bge-base gains (Table 3): for long-tail entities, the 2024 article is typically much richer than the 2018 stub, and the same future-derived entity vectors are used on both the query and passage sides of the dot product. Please repeat the main experiments with entity embeddings and the dictionary rebuilt from the December 2018 dump (or at least restricted to entity texts available in 2018), and report the overlap between the two entity vocabularies and the difference in retrieved answers. The Limitations section should also state this temporal constraint.
  2. [Section 2, Section 3] The central claim that KPR supports dynamic entity-knowledge updates without retraining is not directly verified. The design freezes entity embeddings and uses an updatable dictionary linker, but no experiment changes the entity embeddings or the linker after training and measures retrieval accuracy. Table 2 compares entity-embedding sources, yet it does not state whether the KPR attention layer was retrained for each row; if it was retrained, the table does not support the no-retraining update claim. Please add an experiment that swaps or adds entity embeddings at inference time (with the linker updated accordingly) and reports accuracy on the same test sets, or explicitly clarify the training protocol used in Table 2.
minor comments (4)
  1. [Section 3.1] The sentence "KPR significantly outperforms all baselines across datasets and base models" is contradicted by Table 1: BM25 achieves 71.2 top-20 accuracy on EntityQuestions, higher than KPR_BERT's 69.4. The sentence should be restricted to the neural dense-retriever baselines or to the overall average.
  2. [Appendix C] The statement that KPR "contains 5.6B parameters" is confusing because the vast majority are frozen, non-trainable entity embeddings stored in a lookup table; please give the trainable-parameter count alongside.
  3. [Figure 2] In the provided extracted version of Figure 2, the panels appear to duplicate the same plot with repeated axis labels; please ensure the final figure has one clear y-axis label and no duplicated panels.
  4. [Section 2] The phrase "no-opembedding" should be "no-op embedding".

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: KPR's gains rest on held-out evaluation with frozen entity embeddings, not on a fitted constant or a self-citation chain.

full rationale

We find no derivation step in which a claimed prediction reduces by construction to its own inputs or to a self-citation. KPR's entity embeddings are frozen, precomputed features obtained by BERT inference over Wikipedia anchor passages (Section 2, Appendix C), and they are not optimized on the EQ, NQ, or TQA test sets. The context-entity attention layer is trained on the standard DPR training mixture, and the headline comparisons in Tables 1 and 3 are held-out evaluations with confidence intervals over multiple runs. The April 2024 entity embeddings versus December 2018 retrieval corpus noted in Appendix C is a genuine external-validity and potential-contamination concern, but it is not circularity: it does not make the reported EQ gain equal to a fitted parameter or to the method's input by construction. Self-citations (e.g., LUKE for multi-token position averaging, Wikipedia2Vec as an embedding baseline) are design details or baselines, not load-bearing justifications of the central claim. No uniqueness theorem is imported from the authors' prior work, and no known result is merely renamed. Under the stated criteria, the correct finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

KPR does not postulate new external entities or forces. The no-op embedding is a learned component, not a new entity. The main assumptions are about the utility of BERT-derived entity embeddings and the dictionary linker, plus the unstated temporal compatibility of the knowledge base and corpus.

free parameters (4)
  • Entity linker thresholds = link probability > 5%, commonness > 30%
    Chosen by hand to filter entity names from Wikipedia hyperlinks; affects which entities are detected and thus all downstream results.
  • Sigmoid length-bias constant = 1
    The activation sigmoid(x - log N + 1) is adopted as a design choice and shown to outperform softmax in ablations.
  • Max passages per entity for embeddings = 128
    Randomly selected up to 128 Wikipedia passages per entity; a computational limit that could affect embedding quality.
  • bge-base training hyperparameters = 20 epochs, learning rate 5e-5, temperature 0.02, cosine similarity
    Tuned on NQ development set, which is legitimate, but these values affect the reported state-of-the-art comparison.
assumptions (4)
  • domain assumption BERT's [CLS] embedding and [MASK] token embeddings provide useful representations for retrieval and entity knowledge.
    The entire method depends on these embeddings being informative; no independent evidence is provided beyond the experimental outcome.
  • domain assumption Wikipedia hyperlinks and anchor text are a reliable source of entity names and referents.
    The dictionary-based entity linker is built from Wikipedia hyperlinks, and all candidate entities are kept without disambiguation.
  • domain assumption The attention mechanism can learn to select relevant entities based on query context.
    The paper assumes that a single-head attention layer over entity embeddings is sufficient to capture entity relevance.
  • ad hoc to paper Entity embeddings computed from the April 2024 Wikipedia dump are compatible with a December 2018 passage corpus.
    The paper does not justify the temporal mismatch, which is a load-bearing assumption for the validity of the comparisons.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic Injection of Entity Knowledge into Dense Retrievers." pith.science (2026). https://pith.science/paper/H3EQ2AEE

@misc{pith2026250703922,
  author       = {Pith},
  title        = {Pith review of: Dynamic Injection of Entity Knowledge into Dense Retrievers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3EQ2AEE}},
  note         = {Machine review of arXiv:2507.03922}
}
read the original abstract

Dense retrievers often struggle with queries involving less-frequent entities due to their limited entity knowledge. We propose the Knowledgeable Passage Retriever (KPR), a BERT-based retriever enhanced with a context-entity attention layer and dynamically updatable entity embeddings. This design enables KPR to incorporate external entity knowledge without retraining. Experiments on three datasets demonstrate that KPR consistently improves retrieval accuracy, with particularly large gains on the EntityQuestions dataset. When built on the off-the-shelf bge-base retriever, KPR achieves state-of-the-art performance among similarly sized models on two datasets. Models and code are released at https://github.com/knowledgeable-embedding/knowledgeable-embedding.

Figures

Figures reproduced from arXiv: 2507.03922 by the authors.

Figure 1
Figure 1. Architecture of KPR, a dense retriever with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Top 20 retrieval accuracy of KPRBERT, DPRPELT, DPRBERT, and BM25 on EQ, grouped into 10 bins based on entity frequency in Wikipedia. Model EQ NQ TQA Avg contriever (Izacard et al., 2022) 63.0 67.9 73.9 68.3 gte-base (Li et al., 2023) 70.8 77.0 76.6 74.7 e5-base (Wang et al., 2022) 72.4 86.2 81.4 80.0 bge-base (Xiao et al., 2024) 71.0 82.3 80.0 77.8 KPRbge-base 76.8 82.4 81.5 80.2 [PITH_FULL_IMAGE:figures/full_fig_p… view at source ↗
Figure 3
Figure 3. Qualitative analysis of KPR’s attention mechanism on the GraphQuestions dataset. Entity names are [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 16 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael L...

    arXiv 2024

  4. [4]

    Tom Ayoola, Shubhi Tyagi, Joseph Fisher, Christos Christodoulopoulos, and Andrea Pierleoni. 2022. https://doi.org/10.18653/v1/2022.naacl-industry.24 R e F in ED : An efficient zero-shot-capable approach to end-to-end entity linking . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human...

    doi:10.18653/v1/2022.naacl-industry.24 2022

  5. [5]

    Petr Baudi s and Jan S ediv\' y . 2015. https://doi.org/10.1007/978-3-319-24027-5_20 Modeling of the question answering task in the Y oda QA system . In Proceedings of the 6th International Conference on Experimental IR Meets Multilinguality, Multimodality, and Interaction, page 222–228

    work page doi:10.1007/978-3-319-24027-5_20 2015

  6. [6]

    Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. https://aclanthology.org/D13-1160/ Semantic parsing on F reebase from question-answer pairs . In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533--1544

    work page 2013

  7. [7]

    Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. https://openreview.net/forum?id=5k8F6UU39V Autoregressive entity retrieval . In International Conference on Learning Representations

    work page 2021

  8. [8]

    Shubham Chatterjee, Iain Mackie, and Jeff Dalton. 2024. https://doi.org/10.1007/978-3-031-56027-9_13 DREQ : Document re-ranking using entity-based query understanding . In Advances in Information Retrieval, pages 210--229

    doi:10.1007/978-3-031-56027-9_13 2024

Show all 51 references
  1. [9]

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. 2022. https://arxiv.org/abs/2209.10652v1 Toy models of superposition . arXiv preprint arXiv:2209.10652v1

    2022 arXiv

  2. [10]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. https://proceedings.mlr.press/v119/guu20a.html Retrieval augmented language model pre-training . In Proceedings of the 37th International Conference on Machine Learning, pages 3929--3938

    2020

  3. [11]

    Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020. https://doi.org/10.5281/zenodo.1212303 spa C y: Industrial-strength natural language processing in P ython

    2020 doi

  4. [12]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. https://openreview.net/forum?id=jKN1pXi7b0 Unsupervised dense information retrieval with contrastive learning . Transactions on Machine Learning Research

    2022

  5. [13]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  6. [14]

    Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. https://proceedings.mlr.press/v202/kandpal23a/kandpal23a.pdf Large language models struggle to learn long-tail knowledge . In Proceedings of the 40th International Conference on Machine Learning,...

    2023

  7. [15]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. https://arxiv.org/abs/2001.08361v1 Scaling laws for neural language models . arXiv preprint arXiv:2001.08361v1

    2020 arXiv

  8. [16]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  9. [17]

    Diederik P Kingma and Jimmy Ba. 2014. https://arxiv.org/abs/1412.6980v9 Adam: A method for stochastic optimization . arXiv preprint arXiv:1412.6980v9

    2014 arXiv

  10. [18]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  11. [19]

    Hyunji Lee, Luca Soldaini, Arman Cohan, Minjoon Seo, and Kyle Lo. 2023. https://arxiv.org/abs/2311.09765v1 Back to basics: A simple recipe for improving out-of-domain retrieval in dense encoders . arXiv preprint arXiv:2311.09765v1

    2023 arXiv

  12. [20]

    Quentin Lhoest, Albert Villanova del Moral, Patrick von Platen, Thomas Wolf, Mario Šaško, Yacine Jernite, Abhishek Thakur, Lewis Tunstall, Suraj Patil, Mariama Drame, Julien Chaumond, Julien Plu, Joe Davison, Simon Brandeis, Victor Sanh, Teven Le Scao, Kevin Canwen Xu, Nicolas...

    2021

  13. [21]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. https://arxiv.org/abs/2308.03281v1 Towards general text embeddings with multi-stage contrastive learning . arXiv preprint arXiv:2308.03281v1

    2023 arXiv

  14. [22]

    Xitong Liu and Hui Fang. 2015. https://doi.org/10.1007/s10791-015-9267-x Latent entity space: a novel retrieval approach for entity-bearing queries . Inf. Retr., 18(6):473–503

    2015 doi

  15. [23]

    Zhenghao Liu, Chenyan Xiong, Maosong Sun, and Zhiyuan Liu. 2018. https://doi.org/10.18653/v1/P18-1223 Entity-duet neural ranking: Understanding the role of knowledge graph semantics in neural information retrieval . In Proceedings of the 56th Annual Meeting of the Association ...

    2018 doi

  16. [24]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.acl-long.546 When not to trust language models: Investigating effectiveness of parametric and non-parametric memories . In Proceedings of the 6...

    2023 doi

  17. [25]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/6f1d43d5a82a37e89b0665b33bf3a182-Paper-Conference.pdf Locating and editing factual associations in GPT . In Advances in Neural Information Processing Sy...

    2022

  18. [26]

    Rada Mihalcea and Andras Csomai. 2007. https://doi.org/10.1145/1321440.1321475 Wikify! linking documents to encyclopedic knowledge . In Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, page 233–242

    2007

  19. [27]

    David Milne and Ian H. Witten. 2008. https://doi.org/10.1145/1458082.1458150 Learning to link with W ikipedia . In Proceedings of the 17th ACM Conference on Information and Knowledge Management, page 509–518

    2008

  20. [28]

    Thong Nguyen, Shubham Chatterjee, Sean MacAvaney, Iain Mackie, Jeff Dalton, and Andrew Yates. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.45 D y V o: Dynamic vocabularies for learned sparse retrieval with entities . In Proceedings of the 2024 Conference on Empirical Meth...

    2024 doi

  21. [29]

    Jason Ramapuram, Federico Danieli, Eeshan Gunesh Dhekane, Floris Weers, Dan Busbridge, Pierre Ablin, Tatiana Likhomanenko, Jagrit Digani, Zijin Gu, Amitis Shidani, and Russell Webb. 2025. https://openreview.net/forum?id=Zhdhg6n2OG Theory, analysis, and best practices for sigmo...

    2025

  22. [30]

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. https://doi.org/10.1145/3394486.3406703 Deep S peed: System optimizations enable training deep learning models with over 100 billion parameters . In Proceedings of the 26th ACM SIGKDD International Confere...

    2020

  23. [31]

    Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.496 Simple entity-centric questions challenge dense retrievers . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, page...

    2021 doi

  24. [32]

    Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.320 Retrieval augmentation reduces hallucination in conversation . In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784--3803

    2021 doi

  25. [33]

    Daniil Sorokin and Iryna Gurevych. 2018. https://doi.org/10.18653/v1/S18-2007 Mixing context granularities for improved entity linking on question answering data across entity categories . In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, p...

    2018 doi

  26. [34]

    Yu Su, Huan Sun, Brian Sadler, Mudhakar Srivatsa, Izzeddin G \"u r, Zenghui Yan, and Xifeng Yan. 2016. https://doi.org/10.18653/v1/D16-1054 On generating characteristic-rich question sets for QA evaluation . In Proceedings of the 2016 Conference on Empirical Methods in Natural...

    2016 doi

  27. [35]

    Hai Dang Tran and Andrew Yates. 2022. https://doi.org/10.1145/3511808.3557285 Dense retrieval with entity views . In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, page 1955–1964

    2022

  28. [36]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . In Advances in Ne...

    2017

  29. [37]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. https://arxiv.org/abs/2212.03533v2 Text embeddings by weakly-supervised contrastive pre-training . arXiv preprint arXiv:2212.03533v2

    2022 arXiv

  30. [38]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Perric Cistac, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. https://www.aclweb.org/anthology...

    2020

  31. [39]

    Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.519 Scalable zero-shot entity linking with dense entity retrieval . In Proceedings of the 2020 Conference on Empirical Methods in Natural Langu...

    2020 doi

  32. [40]

    Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.35 R etro MAE : Pre-training retrieval-oriented language models via masked auto-encoder . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Proces...

    2022 doi

  33. [41]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. https://doi.org/10.1145/3626772.3657878 C- P ack: Packed resources for general C hinese embeddings . In Proceedings of the 47th International ACM SIGIR Conference on Research and Deve...

    2024

  34. [42]

    Chenyan Xiong, Jamie Callan, and Tie-Yan Liu. 2017 a . https://doi.org/10.1145/3077136.3080768 Word-entity duet representations for document ranking . In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 763–772

    2017

  35. [43]

    Chenyan Xiong, Russell Power, and Jamie Callan. 2017 b . https://doi.org/10.1145/3038912.3052558 Explicit semantic ranking for academic search via knowledge graph embedding . In Proceedings of the 26th International Conference on World Wide Web, page 1271–1279

    2017

  36. [44]

    Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, and Yuji Matsumoto. 2020 a . https://doi.org/10.18653/v1/2020.emnlp-demos.4 W ikipedia2 V ec: An efficient toolkit for learning and visualizing the embeddings of words and entities from ...

    2020 doi

  37. [45]

    Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. 2020 b . https://doi.org/10.18653/v1/2020.emnlp-main.523 LUKE : Deep contextualized entity representations with entity-aware self-attention . In Proceedings of the 2020 Conference on Empirical Metho...

    2020 doi

  38. [46]

    Ikuya Yamada, Koki Washio, Hiroyuki Shindo, and Yuji Matsumoto. 2022. https://doi.org/10.18653/v1/2022.naacl-main.238 Global entity disambiguation with BERT . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: ...

    2022 doi

  39. [47]

    Deming Ye, Yankai Lin, Peng Li, Maosong Sun, and Zhiyuan Liu. 2022. https://doi.org/10.18653/v1/2022.acl-short.57 A simple but effective pluggable entity lookup table for pre-trained language models . In Proceedings of the 60th Annual Meeting of the Association for Computation...

    2022 doi

  40. [48]

    Da Yu, Edith Cohen, Badih Ghazi, Yangsibo Huang, Pritish Kamath, Ravi Kumar, Daogao Liu, and Chiyuan Zhang. 2025. https://arxiv.org/abs/2502.01637v1 Scaling embedding layers in language models . arXiv preprint arXiv:2502.01637v1

    2025

  41. [49]

    Xinyu Zhang, Kelechi Ogueji, Xueguang Ma, and Jimmy Lin. 2023 a . https://doi.org/10.1145/3613447 Toward best practices for training multilingual dense retrieval models . ACM Trans. Inf. Syst., 42(2)

    2023 doi

  42. [50]

    Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. https://doi.org/10.18653/v1/P19-1139 ERNIE : Enhanced language representation with informative entities . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,...

    2019 doi

  43. [51]

    Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.594 Plug-and-play knowledge injection for pre-trained language models . In Proceedings o...

    2023 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.