Pith. sign in

REVIEW 3 major objections 5 minor 22 references

This paper claims that a retriever's blind spots—entities it reliably fails to surface under practical top-k budgets—can be predicted from embedding geometry before indexing, and that targeted knowledge-base augmentation of those entities i

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 02:45 UTC pith:UU45S2PU

load-bearing objection RPS is a sound new measurement construct with clean probe results, but the ARGUS remedy claim is missing the one control that would distinguish targeted selection from generic document expansion. the 3 major comments →

arxiv 2602.09616 v2 pith:UU45S2PU submitted 2026-02-10 cs.IR cs.AI

With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spots

Zeinab Sadat Taghavi , Ali Modarressi , Hinrich Schutze , Andreas Marfurt This is my paper
classification cs.IR cs.AI MSC 68P2068T50
keywords retrieval blind spotsRetrieval Probability Score (RPS)entity retrievabilityRAG robustnesspre-index auditingembedding geometrydocument augmentationWikidata-Wikipedia alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that the failures of neural retrievers in RAG systems are not random noise but systematic blind spots: entities that are relevant to queries yet mapped to inaccessible parts of embedding space, so they are consistently missed under practical top-k budgets. To measure this, the authors define the Retrieval Probability Score (RPS), the expected top-k hit rate of an entity across query contexts derived from Wikidata relations, and show that low-RPS entities occupy separable regions of embedding space. They then train lightweight probes that predict RPS directly from entity embeddings (Pearson r ≈ 0.65–0.80), making blind-spot risk detectable before indexing. Finally, ARGUS augments flagged high-risk entities with defining context from Wikipedia first paragraphs and indexes these views alongside the originals, yielding average absolute gains of +3.4 nDCG@5 and +4.5 nDCG@10 across eight retrievers on BRIGHT, IMPLIRET, and RAR-B. The paper's stake is that preemptive, index-time auditing and remedy of blind spots is a practical path to trustworthy RAG, though its own auxiliary association analysis is explicitly labeled as non-causal.

Core claim

The central claim is that entity-level retrievability is a learnable, geometry-encoded property of neural retrievers, not an irreducible accident of query phrasing. Working from a large random sample of Wikidata–Wikipedia aligned entities, the paper defines RPS_k as the probability that a target entity surfaces in the top-k when ranked against N−1 strictly disjoint neutral entities across its Wikidata-derived related-query set, adopting N=800 for stable audits. Audits show low- and high-RPS entities cluster in distinct embedding regions; from those embeddings, per-retriever probes (ridge, XGBoost, MLP) predict RPS with Pearson correlation ≈0.65–0.80 and semi-classification accuracy ≈0.65–0.8

What carries the argument

The central object is the Retrieval Probability Score (RPS_k): for an entity x, the empirical average, over Wikidata 1-hop related entities used as proxy queries, of an indicator that x ranks in the top k when competing against N−1 neutral entities that share no direct Wikidata link with the query. Work it does: (1) it converts blind spot from a vague failure mode into a measurable, per-entity probability; (2) with N=800 it decouples genuine geometric retrievability from random hits (the chance baseline k/N at k=50 is 0.0625); and (3) its empirical values serve as labels for diagnostic probes that map an entity embedding to predicted RPS, making pre-index detection possible. ARGUS is the sec

Load-bearing premise

The load-bearing premise is that the targeted selection of high-risk entities by RPS—not the act of appending extra defining text—causes the retrieval gains; the paper compares original vs ARGUS-augmented indexes but never tests augmenting a random or complete set of entities, and the paper itself notes (Section 8) that its uniform configuration may be suboptimal and that the augmentation form and threshold likely depend on retriever and corpus.

What would settle it

Run ARGUS's augmentation procedure on a random subset of entities (or on all entities) using the same Wikipedia-first-paragraph passages and the same indexing setup; if the nDCG gains match or exceed ARGUS's, the RPS diagnosis stage is not the active ingredient. Separately, train a probe on one retriever's RPS audit and apply it to another retriever's embeddings; if the ranking of high-risk entities collapses, the claim that blind-spot risk is a generalizable geometric property needs qualification.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Blind-spot risk can be audited before deployment: probes predict RPS from entity embeddings with Pearson r ≈ 0.65–0.80, so a corpus can be screened for high-risk entities without running retrieval over it.
  • Index-time augmentation of flagged entities improves downstream ranking: average absolute gains of +3.4 nDCG@5 and +4.5 nDCG@10 across eight retrievers on BRIGHT, IMPLIRET, and RAR-B, with larger suite-level gains (up to +6.76 on IMPLIRET for Document Expansion).
  • The two remedies trade off: Document Expansion is the most consistent in aggregate, while LLM Synthesis adds one view per document but shows higher variance across retrievers (e.g., it degrades JINA-V3 on IMPLIRET while expansion improves it).
  • Blind spots persist across architectures, including reasoning-oriented retrievers like REASONIR-8B, so no tested retriever is exempt from entity-level geometric failure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A critical control is missing in the paper: augmenting a random (or complete) subset of entities with the same Wikipedia-first-paragraph passages would distinguish the diagnosis stage's contribution from a plain document-expansion effect; the current design cannot rule out that the gains come from the added text itself, not from RPS-based selection.
  • If the geometry-to-risk mapping is transferable, the same probes could be used to audit new corpora or new retrievers without retraining; the paper does not test this transfer across embedding spaces.
  • RPS characterizes entities, not downstream queries; using it to decide which mentions to expand in entity-centric QA, or to curate corpus content for a target retriever, could affect generator groundedness in ways the paper does not measure.
  • The fixed global threshold τ=0.3 is an operating point chosen once; per-domain or per-retriever thresholds would likely change which entities get flagged, and the paper leaves adaptive thresholding open.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces the Retrieval Probability Score (RPS), a metric that measures how often an entity appears in the top-k of controlled retrieval pools constructed from Wikidata relations and Wikipedia first paragraphs. Using RPS, the authors show that entities cluster into low/mid/high retrievability regions in embedding space, train lightweight supervised probes to predict RPS from entity embeddings, and propose ARGUS, a pre-index pipeline that flags low-RPS entities via a threshold and augments their documents with Wikipedia first-paragraph context using either document expansion or LLM synthesis. Experiments on BRIGHT, IMPLIRET, and RAR-B with eight retrievers report consistent nDCG@5/10 improvements over an unaugmented baseline.

Significance. If the causal claim behind ARGUS is correct, the paper offers a practical and novel pre-index auditing and remedy tool: blind-spot risk can be estimated offline from embeddings, and targeted augmentation can improve downstream retrieval without retraining or query rewriting. The RPS construction is careful (KG-disjointness, neutral-pool size analysis, mention-aware pooling), the probe results are reported on held-out test splits with validation-based hyperparameter selection, and the full per-task tables and code/data availability strengthen reproducibility. However, the central remedy claim is currently under-supported: the experiments do not isolate the contribution of RPS-guided selection from the generic effect of adding Wikipedia text to documents. The predictability result, while plausible, is also less surprising than presented because RPS is defined using the same embeddings that the probe consumes. With an appropriate random/all-entity augmentation control, the paper's contribution could be solid; without it, the main conclusion is not yet established.

major comments (3)
  1. [§7.1, Table 2; §5.2] The end-to-end claim that RPS-guided targeting drives the nDCG gains is not supported by the experimental design. ARGUS is always compared against Original indexing, and ARGUS always injects Wikipedia first paragraphs for flagged entities. There is no control that augments all NER-extracted entities, or a random subset with a matched number of additional views, using the same kAug and reference KB. If random or all-entity augmentation produces similar nDCG gains, the headline conclusion that 'preemptively remedying blind spots is critical' collapses into a standard document-expansion effect. Please add these control conditions and report the comparison in Table 2.
  2. [§4.1, Table 1; §6; §7.1] There is a mismatch between the retrieval budget used for risk diagnosis and the budgets used for downstream evaluation. Table 1 calibrates probes at k=50, but Table 2 reports nDCG@5/10. If ARGUS uses the k=50 probe to flag entities for a k=5/10 evaluation, the threshold τ=0.3 does not directly correspond to blind spots at the evaluation cutoff; an entity with RPS@50<0.3 can still be retrieved at k=5, and an entity with RPS@50>0.3 can be missed at k=5. The paper should either train and apply probes at the same k as the evaluation, or provide evidence that risk at k=50 transfers to k=5/10.
  3. [§3.2, §4.1; Appendix E.1] The claim that blind-spot risk is 'geometrically encoded' and can be predicted 'pre-index' is partially definitional. RPS is computed from cosine similarities among embeddings produced by the same encoder, and the probe is a supervised regressor from those same embeddings to the resulting RPS. A correlation of r≈0.65–0.80 is therefore expected if the mapping from embedding geometry to rank-based hit probability is smooth; it does not by itself establish that blind spots are intrinsic, training-induced properties of entities. The Appendix E.1 disclaimer that the association analysis 'is not intended to establish causality' is appropriate, but the main text and abstract present the geometric predictability as a stronger structural finding. I recommend either tempering the causal/intrinsic language or adding a control such as training a probe on one retriever's embeddings and testing on ano
minor comments (5)
  1. [Table 4] The retriever name 'textscEeason-Embed' appears to be a typo; should be 'REASON-EMBED'.
  2. [Table 5] Several rows incorrectly read 'ARAGUS' (e.g., BGE-M3, CONTRIEVER rows); should be 'ARGUS'.
  3. [§5] The sentence 'requiring neither query rewriting nor expensive retrieval evaluation over the target corpus..' has a doubled period and could be split for clarity.
  4. [§3.2] The RPS definition and top-k hit equations are not numbered. Numbering them would help the reader refer to the formal claims in later sections.
  5. [Figure 3 caption] The caption says 'successful retrieval rates match the chance regime (k/N≈0.5)' — this is correct only for k=50, N=100; please make the values explicit.

Circularity Check

0 steps flagged

No significant circularity: RPS is an empirically measured label and the probe is a supervised estimator, not a constructional identity.

full rationale

The claimed derivation chain is not circular. RPS (Section 3.2) is defined as an expected top-k hit probability computed from the retriever's own cosine rankings over Wikidata-derived query/neutral pools; the diagnostic probes (Section 4.1) are supervised regressors trained on (entity embedding, empirical RPS) pairs. Predicting RPS from the embedding is a legitimate statistical shortcut: the embedding is available at index time, while RPS requires retrieval simulations. There is no equation in which a predicted quantity is defined in terms of the quantity it is said to predict, and no fitted parameter is renamed as a prediction. The LDA separability of RPS-labeled embeddings is an empirical observation, not an input assumption. The main weakness is causal attribution, not circularity: Section 7.1 compares Original indexing only against ARGUS, which always injects Wikipedia first-paragraph text for flagged entities, so the nDCG gains could in principle be a generic document-expansion effect rather than evidence that RPS-based selection is 'critical'; the paper itself cautions in Appendix E.1 that its association analysis 'is not intended to establish causality.' That gap warrants a random-/all-entity augmentation ablation, but it is a confound, not a reduction of the output to the input by construction. Self-citations (Taghavi et al. 2025; Modarressi et al. 2025) are used for related work and one evaluation benchmark; the central findings also hold on BRIGHT and RAR-B, so no load-bearing self-citation chain drives the result.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 2 invented entities

The paper introduces one operational construct (RPS) and one inferred property (structural blind spots), plus a diagnosis-to-remedy pipeline. It has no fitted scientific constants, but hand-chosen operational parameters (τ=0.3, kAug=2, N=800, k=50) govern all results. Its key unstated premises are the proxy validity of Wikidata-derived queries and the untested assumption that the diagnosis stage — not the injected Wikipedia text — is what improves retrieval on the benchmarks.

free parameters (5)
  • Risk threshold τ = 0.3
    Global cutoff for flagging high-risk entities (predicted RPS < τ), set just below the low-RPS tercile cutoff and never tuned per benchmark; the paper concedes it may be suboptimal per domain (§5.1, Section 8).
  • Reference KB passages per entity kAug = 2
    Number of BM25S-retrieved Wikipedia first paragraphs appended per flagged entity; chosen for lightweight augmentation, not optimized (§5.2).
  • Neutral pool size N = 800
    Selected after inspecting RPS-versus-N stability curves (plateau beyond N≈400, Figure 3); a defensible audit choice but still post hoc.
  • Audit retrieval budget k = 50
    User-defined top-k for the audit and probes; RPS is defined relative to it and results shift with k (sensitivity analysis, Figure 5).
  • Probe hyperparameters = per-retriever validation selection
    Ridge/XGBoost/MLP hyperparameters selected by validation RMSE (§4.1, Appendix C.1); standard model selection, not fitted to test labels.
axioms (6)
  • domain assumption Wikidata 1-hop neighbors with Wikipedia-first-paragraph encodings are a valid query distribution for entity retrievability.
    The entire RPS audit (§3.1) treats related Wikidata entities as proxy queries; real RAG queries may differ in phrasing and intent.
  • domain assumption Excluding only direct 1-hop KG neighbors of the query (while the target is itself such a neighbor) suffices as a strictly disjoint neutral control.
    Appendix B.3: neutrality is defined only against the query entity, not against the target; indirect relations are not controlled, so the 'controlled assessment of geometric robustness' is only partially controlled.
  • domain assumption Span-pooled paragraph embeddings are faithful entity representations across token-level and sentence-level retrievers.
    Appendix B.4 defines g(·) as a per-retriever pooling operator; for sentence-level models the span is ignored, so entity and paragraph share a single vector.
  • domain assumption dslim/bert-base-NER extracts the entities relevant to benchmark queries.
    ARGUS diagnosis (§5.1) depends on this off-the-shelf tagger; unrecognized entities are never augmented or remedied.
  • domain assumption Wikipedia first paragraphs transfer as a reference KB for benchmark documents.
    Section 5.2 uses Wikipedia first paragraphs as retrieval/augmentation source while benchmark documents come from other domains; the matching may be weaker outside Wikipedia-like text.
  • standard math Supervised probe regression is well-posed (no leakage between RPS label computation and probe input).
    Section 4.1 trains on entity embeddings labeled by empirically computed RPS; both derive from the same encoder and the same paragraph text, so the probe partly learns the retriever's own geometry-to-ranking map.
invented entities (2)
  • Retrieval Probability Score (RPS) metric no independent evidence
    purpose: Quantify entity-level retrievability as the expected top-k hit rate over Wikidata-derived related-query contexts against disjoint neutral pools.
    A measurement construct, not a physical entity; its validity rests on the proxy-query and neutral-pool assumptions, and the paper does not validate RPS against an external human-labeled retrievability criterion.
  • Blind spot (low-RPS entity) no independent evidence
    purpose: Designate entities that are systematically missed by a retriever under a fixed top-k budget.
    Defined operationally as low RPS; the claim that blind spots are a structural phenomenon rather than an artifact of embedding-space anisotropy (norm/density effects) is not tested against such confounds.

pith-pipeline@v1.3.0-alltime-deepseek · 35325 in / 20452 out tokens · 205381 ms · 2026-08-03T02:45:29.927763+00:00 · methodology

0 comments
read the original abstract

Reliable retrieval-augmented generation (RAG) systems depend fundamentally on the retriever's ability to find relevant information. We show that neural retrievers used in RAG systems have blind spots, which we define as the failure to retrieve entities that are relevant to the query, but have low similarity to the query embedding. We investigate the training-induced biases that cause such blind spot entities to be mapped to inaccessible parts of the embedding space, resulting in low retrievability. Using a large-scale dataset constructed from Wikidata relations and first paragraphs of Wikipedia, and our proposed Retrieval Probability Score (RPS), we show that blind spot risk in standard retrievers (e.g., CONTRIEVER, REASONIR) can be predicted pre-index from entity embedding geometry, avoiding expensive retrieval evaluations. To address these blind spots, we introduce ARGUS, a pipeline that enables the retrievability of high-risk (low-RPS) entities through targeted document augmentation from a knowledge base (KB), first paragraphs of Wikipedia, in our case. Extensive experiments on BRIGHT, IMPLIRET, and RAR-B show that ARGUS achieves consistent improvements across all evaluated retrievers (averaging +3.4 nDCG@5 and +4.5 nDCG@10 absolute points), with substantially larger gains in challenging subsets. These results establish that preemptively remedying blind spots is critical for building robust and trustworthy RAG systems.

Figures

Figures reproduced from arXiv: 2602.09616 by Ali Modarressi, Andreas Marfurt, Hinrich Schutze, Zeinab Sadat Taghavi.

Figure 1
Figure 1. Figure 1: Retrieval Probability Score (RPS) computation and retriever blind-spot analysis. (Top) Evaluation pipeline: (1) construct a Wikidata–Wikipedia aligned dataset, (2) build query￾specific retrieval sets with strictly disjoint neutral entities, and (3) compute RP S from retrieval consistency. (Bottom) Average RPS over a large random entity sample at k = 50 with N = 800 neutrals (suppressing chance hits). Stand… view at source ↗
Figure 2
Figure 2. Figure 2: LDA projections of entity embeddings labeled by RPS terciles (low/mid/high) at k = 50 under increasing neutral pool sizes N, comparing a low-RPS retriever (BGE-M3) to a high-RPS retriever (REASONIR-8B). Robust retrievers retain denser high-RPS regions (blue) as N grows, indicating higher expected top-k retrievability for a random entity, while persistent low-RPS regions (red) across models confirm intrinsi… view at source ↗
Figure 3
Figure 3. Figure 3: Impact of neutral pool size (N) on fraction of entities with RPSk > 0.5 (k = 50). At N = 100, successful retrieval rates match the chance regime (k/N ≈ 0.5). Beyond N ≥ 400, curves decouple from chance and plateau, revealing stable, model￾specific behavior. Hence, we adopt N = 800, so that high RPS reflects genuine geometric retrievability. Intuitively, RPSk(x | wx) represents the probability of dis￾covery… view at source ↗
Figure 4
Figure 4. Figure 4: The ARGUS Pipeline: Diagnosis and Remedy of Geometric Blind Spots. (A) Diagnosis: The system first extracts named entities and predicts their retrievability (RP Sk) using the target retriever. Entities falling below the safety threshold (RP S < τ ) are flagged as blind spots (high-risk) located in inaccessible regions of the embedding space. (B) Augmentation: To remedy these blind spots, ARGUS retrieves de… view at source ↗
Figure 5
Figure 5. Figure 5: Sensitivity of retrieval consistency to the retrieval window size (k) at fixed N = 800. Increasing the user-defined parameter k expands the retrieval scope. Standard retrievers (e.g., Contriever, BGE-M3) exhibit approximately linear growth consis￾tent with statistical scaling of the random-hit window. In contrast, ReasonIR displays a non-linear trajectory with a mild elbow, in￾dicating that its gains are d… view at source ↗
Figure 6
Figure 6. Figure 6: Extended geometric visualization of two-dimensional LDA projections for all evaluated retrievers under increasing neutral pool sizes N. Consistent with the main analysis, entities are labeled by RPS terciles (low/mid/high) at k = 50. The full benchmark reveals that standard dense retrievers (e.g., BGE-M3, Qwen3, GritLM) exhibit a collapsing geometric structure similar to Contriever, where low-RPS regions d… view at source ↗
Figure 7
Figure 7. Figure 7: Calibration analysis of embedding-based diagnostic probes (Predicted vs. Empirical RPS). (Top) Prediction density: Heatmaps of predicted versus true RPS illustrate that probes recover the overall retrievability structure. For standard retrievers (e.g., Contriever), the concentration near low RPS reflects the skew toward geometrically hard-to-retrieve entities (blind spots). In contrast, ReasonIR shows a mo… view at source ↗
Figure 8
Figure 8. Figure 8: Prompt template used for KB-guided LLM synthesis in ARGUS. Candidate entities are paired with retrieved Wikipedia first-paragraph contexts; the model inserts short comma-delimited descriptors only when the context check passes. Where space permits, we additionally include Recall@k at matching cutoffs to highlight changes in coverage at larger retrieval windows. These extended-cutoff results are con￾sistent… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 6 linked inside Pith

  1. [1]

    Read the ‘Input Text’

  2. [2]

    Review the list of ‘Candidate Entities’ and their provided Wikipedia context

  3. [3]

    -> ACTION: Do NOT augment

    For EACH entity, perform a Context Check: -Does the entity in the ‘Input Text’ refer to the exact same concept described in the Wikipedia context? -Example of MISMATCH: Input says ‘it was half past nine’ (time), but Wikipedia describes ‘Half Past Nine’ (the album). -> ACTION: Do NOT augment. -Example of MATCH: Input says ‘St. Peter’, Wikipedia describes ‘...

  4. [4]

    If (and ONLY if) an entity passes the Context Check, insert a short description (max 5 words, enclosed in commas) immediately after the entity

    Generation Step: Rewrite the Input Text. If (and ONLY if) an entity passes the Context Check, insert a short description (max 5 words, enclosed in commas) immediately after the entity

  5. [5]

    If an entity fails the check or you are uncertain, leave it exactly as it is

  6. [6]

    The short description must be a concise type/role/category derived from the Wikipedia context (e.g., ‘industrial EBM album’, ‘Swiss Catholic church in Zurich’), not a long story or extra sentence

  7. [7]

    Nogueira, R

    URL https://openreview.net/forum? id=BC4lIvfSzv. Nogueira, R. and Cho, K. Passage re-ranking with BERT. arXiv preprint arXiv:1901.04085, 2019. Robertson, S. and Zaragoza, H. The probabilistic relevance framework: Bm25 and beyond.Found. Trends Inf. Retr., 3 (4):333–389, April 2009. ISSN 1554-0669. doi: 10.1561/ 1500000019. URL https://doi.org/10.1561/ 1500...

    Pith/arXiv arXiv 1901

  8. [8]

    If the same entity appears multiple times in the Input Text, augment only the first occurrence and leave the others unchanged

  9. [9]

    Your entire output MUST be only the fully augmented text, with no headings, labels, explanations, or surrounding formatting. ### Candidate Entities & Context: [ENTITY_1] : [WIKI_FIRST_PARAGRAPH_1] [ENTITY_2] : [WIKI_FIRST_PARAGRAPH_2] … ### Input Text: [ORIGINAL_DOCUMENT_TEXT] ### Final Annotated Text: Figure 8.Prompt template used for KB-guided LLM synth...

  10. [10]

    Taghavi, Z

    URL https://openreview.net/forum? id=ykuc5q381b. Taghavi, Z. S., Modarressi, A., Ma, Y ., and Schuetze, H. ImpliRet: Benchmarking the implicit fact retrieval chal- lenge. In Christodoulopoulos, C., Chakraborty, T., Rose, C., and Peng, V . (eds.),Proceedings of the 2025 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 33168–33190, Su...

    2025

  11. [11]

    ISBN 979-8-89176-332-6

    Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main

    doi:10.18653/v1/2025.emnlp-main 2025

  12. [20]

    Do not introduce new entities or facts that are not supported by the Wikipedia context

    Do not change the wording, order, or punctuation of the Input Text, except for inserting these short descriptions. Do not introduce new entities or facts that are not supported by the Wikipedia context

  13. [148]

    eacl-main.148/

    URL https://aclanthology.org/2023. eacl-main.148/. Muennighoff, N., SU, H., Wang, L., Yang, N., Wei, F., Yu, T., Singh, A., and Kiela, D. Generative repre- sentational instruction tuning. InThe Thirteenth In- ternational Conference on Learning Representations,

    2023

  14. [272]

    naacl-main.272/

    URL https://aclanthology.org/2022. naacl-main.272/. Shao, R., Qiao, R., Kishore, V ., Muennighoff, N., Lin, X. V ., Rus, D., Low, B. K. H., Min, S., tau Yih, W., Koh, P. W., and Zettlemoyer, L. Reasonir: Training retrievers for reasoning tasks.arXiv preprint arXiv:2504.20595, 2025. Shuster, K., Poff, S., Chen, M., Kiela, D., and We- ston, J. Retrieval aug...

    Pith/arXiv arXiv 2022

  15. [320]

    findings-emnlp.320/

    URL https://aclanthology.org/2021. findings-emnlp.320/. Sturua, S., Mohr, I., Akram, M. K., G¨unther, M., Wang, B., Krimmel, M., Wang, F., Mastrapas, G., Koukounas, A., Koukounas, A., Wang, N., and Xiao, H. jina-embeddings- v3: Multilingual embeddings with task lora.arXiv preprint arXiv:2409.10173, 2024. SU, H., Yen, H., Xia, M., Shi, W., Muennighoff, N.,...

    Pith/arXiv arXiv 2021

  16. [322]

    emnlp-main.322/

    URL https://aclanthology.org/2023. emnlp-main.322/. Mallen, A., Asai, A., Zhong, V ., Das, R., Khashabi, D., and Hajishirzi, H. When not to trust language mod- els: Investigating effectiveness of parametric and non- parametric memories. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.),Proceedings of the 61st Annual Meet- ing of the Association for C...

    2023

  17. [550]

    emnlp-main.550/

    URL https://aclanthology.org/2020. emnlp-main.550/. Lee, C., Roy, R., Xu, M., Raiman, J., Shoeybi, M., Catan- zaro, B., and Ping, W. NV-embed: Improved techniques for training LLMs as generalist embedding models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview. net/forum?id=lgsyLSsDRe. Lewis, P., Perez,...

    Pith/arXiv arXiv 2020

  18. [1685]

    emnlp-main.1685/

    URL https://aclanthology.org/2025. emnlp-main.1685/. Team, Q. Qwen3 technical report, 2025. URL https: //arxiv.org/abs/2505.09388. Thakur, N., Reimers, N., R ¨uckl´e, A., Srivastava, A., and Gurevych, I. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In 10 With Argus Eyes: Assessing Retrieval Gaps via Uncertainty...

    Pith/arXiv arXiv 2025

  19. [2021]

    Thakur, N., Bonifacio, L., Fr¨obe, M., Bondarenko, A., Ka- malloo, E., Potthast, M., Hagen, M., and Lin, J

    URL https://openreview.net/forum? id=wCu6T5xFjeJ. Thakur, N., Bonifacio, L., Fr¨obe, M., Bondarenko, A., Ka- malloo, E., Potthast, M., Hagen, M., and Lin, J. System- atic evaluation of neural retrieval models on the touch´e 2020 argument retrieval subset of beir. InProceedings of the 47th International ACM SIGIR Conference on Re- search and Development in...

    Pith/arXiv arXiv 2020

  20. [2025]

    Muennighoff, N., Tazi, N., Magne, L., and Reimers, N

    URL https://openreview.net/forum? id=0OshX1hiSa. Muennighoff, N., Tazi, N., Magne, L., and Reimers, N. MTEB: Massive text embedding benchmark. In Vla- chos, A. and Augenstein, I. (eds.),Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pp. 2014–2037, Dubrovnik, Croatia, May 2023. Association for...

    doi:10.18653/v1/2023.eacl-main 2014

  21. [3938]

    Hong, K., Troynikov, A., and Huber, J

    PMLR, 2020. Hong, K., Troynikov, A., and Huber, J. Context rot: How increasing input tokens impacts llm performance. Technical report, Chroma, July 2025. URL https: //research.trychroma.com/context-rot. Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., and Grave, E. Unsupervised dense in- formation retrieval with contrastive le...

    2020

  22. [8856]

    J¨arvelin, K

    URL https://openreview.net/forum? id=jKN1pXi7b0. J¨arvelin, K. and Kek¨al¨ainen, J. Cumulated gain-based evalu- ation of ir techniques.ACM Transactions on Information Systems (TOIS), 20(4):422–446, 2002. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y ., Ishii, E., Bang, Y . J., Madotto, A., and Fung, P. Survey of halluci- nation in natural language g...

    2002