Pith. sign in

REVIEW 5 major objections 5 minor 20 references

When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gradient norms computed without any queries can tell a dense retriever when a new corpus will make it fail, and when retraining is actually worth it.

desk verdict A legitimate new task with a plausible signal, but the document-only gradient norm may miss exactly the query-side shifts that motivate it; worth reviewing, needs revision. read the letter →

arxiv 2506.01877 v1 pith:CC3WETSP submitted 2025-06-02 cs.IR cs.CL

Dayoon Ko , Jinyoung Kim , Sohyeon Kim , Jinhyuk Kim , Jaehoon Lee , Seonghak Song , Minyoung Lee , Gunhee Kim This is my paper
classification cs.IRcs.CL
keywords out-of-distributiondetectiondenseretrievalgradientnormretrieverupdateevolvingcorporaunsupervisedOODBEIRbenchmarkcontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dense retrievers degrade when the documents they index drift away from their training distribution, but deciding when to retrain is usually guesswork. This paper proposes treating that decision as a prediction problem: before indexing a new corpus, estimate whether it is out-of-distribution for the current retriever. The proposed signal, GradNormIR, is the L2 norm of the retriever's contrastive-loss gradient computed with each document as its own query, using dropout perturbation, top-k nearest neighbors as positives, and hard negatives. On the BEIR benchmark the paper shows that documents flagged by this signal are indeed retrieved less often by their relevant queries, that the signal selects the best retriever among candidates, and that retraining only on flagged corpora keeps retrieval quality near the oracle while beating both never-updating and always-updating baselines. The underlying discovery is that a retriever's own parameter sensitivity, measured through gradient norms, can substitute for labeled queries when judging its readiness for new data.

What carries the argument

The central object is the GradNormIR score, defined for a document d as the average over p positive samples of the L2 norm of the InfoNCE loss gradient with respect to the retriever parameters: $\frac{1}{p}\sum_{i=1}^p \|\nabla_\theta L\|_2$, where the loss is computed treating d as the query, its dropout-masked representation as the query embedding, top-k retrieved documents as the positive candidate pool, and hard negatives drawn from the remaining documents. This object carries the argument because it converts the question 'is this corpus safe to index?' into a measurable property of the retriever itself: high gradient norm means the retriever's parameters are highly sensitive to the document, which the paper links empirically to poor retrieval of that document. The corpus-level OOD ratio $r(C) = |\tilde{C}|/|C|$ is the fraction of documents whose GradNormIR exceeds an in-domain threshold, and a corpus is declared OOD when $r(C) > \gamma$, triggering selective retraining.

What would settle it

Take a dense retriever trained on one domain, construct a new corpus on a completely unrelated topic, and compute GradNormIR for each document; if the flagged high-gradient documents are retrieved by their relevant queries at the same rate as the low-gradient documents (i.e., DRR is not lower for the flagged set), then the gradient norm is not measuring retrieval failure, and the central claim is falsified. A simpler synthetic check: randomly permute document embeddings so that k-NN pseudo-labels become arbitrary, and verify whether the inverse gradient-norm/d2q-recall relationship disappears.

Watch

Extended reading notes

Core claim

The central claim is that the gradient norm of the InfoNCE loss, computed entirely from the retriever's own embeddings without any queries, is inversely related to how well the retriever will retrieve a document. For each document in a new corpus, GradNormIR treats the document as a query, perturbs its representation with dropout, gathers the top-k nearest documents as positive candidates and hard negatives, computes a contrastive loss, and averages the L2 gradient norms over the positives. A document whose gradient norm exceeds a threshold set by in-domain Natural Questions documents is predicted OOD. The paper demonstrates three consequences: flagged documents have lower document retrieval rates across four retrievers than baselines using layer-wise anomaly scores, incremental quantization, or LLM-generated pseudo-queries; choosing the retriever with the lowest OOD ratio recovers close to oracle performance; and a continual-update simulation in which only OOD corpora trigger retraining maintains recall near the upper bound while outperforming the naive strategy of retraining every session. The author's conclusion is that gradient norms of the contrastive loss provide a query-free, unsupervised gauge of when a dense retriever needs updating.

Load-bearing premise

The method assumes that the retriever's own similarity-based pseudo-labels define a meaningful contrastive task for every document, including documents that are truly out-of-distribution and may have no relevant neighbors in the new corpus.

Editorial extensions

If this is right

  • Documents flagged as OOD by GradNormIR are retrieved less often by their relevant queries, with the largest DRR reductions on DBPedia-Entity and Scidocs for BGE.
  • Selecting the retriever with the lowest OOD ratio matches the oracle retriever on most BEIR datasets and is never more than a few Recall@100 points below it.
  • Retraining only on predicted OOD corpora keeps average Recall@100 stable near the upper bound across sessions and surpasses the naive always-retrain baseline from session S6 onward.
  • Removing documents flagged as OOD from a corpus raises Recall@100 summed over the ten BEIR datasets by 12 to 62 points depending on the retriever.
  • GradNormIR values fall into quartiles that show a monotone inverse relationship with document-to-query recall, supporting the interpretation that the gradient norm measures generalizability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper shows 10% document sampling leaves DRR nearly unchanged, GradNormIR could plausibly run as a lightweight streaming monitor on small random batches of an incoming corpus, flagging a distributional shift as soon as the moving average of gradient norms crosses threshold.
  • The method's dependence on an in-domain anchor (Natural Questions) means the threshold is retriever-specific; a testable extension would derive the threshold from the retriever's training corpus itself, or from a held-out validation set, making the approach usable for retrievers whose original training data is not public.
  • The k-NN pseudo-labeling assumption is strongest when the new corpus shares vocabulary or topics with training data; for radically novel domains, gradient norm may partly measure embedding-space sparsity, so a stress test on synthetic disjoint-domain corpora would clarify how much of the signal is OOD-ness vs. k-NN instability.
  • The dropout perturbation at rate 0.02 suggests an ensemble view: averaging gradient norms over several dropout masks might give a lower-variance estimate of the same signal, an extension the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a new task: predicting, before indexing, whether a new document corpus is out-of-distribution (OOD) for a given dense retriever, in order to decide when to update the retriever. The proposed method, GradNormIR, treats each document as a query, forms pseudo-positives via the retriever's own k-NN similarities, adds hard negatives, applies dropout to the document query representation, and computes the gradient norm of an InfoNCE loss with respect to the retriever parameters. Documents with gradient norms above a reference threshold (derived from in-domain NQ documents) are flagged as OOD, and a corpus is declared OOD when the fraction of flagged documents exceeds a threshold gamma. The method is evaluated on ten BEIR datasets with four dense retrievers in three settings: OOD document detection measured by Document Retrieval Rate (DRR), zero-shot best-retriever selection, and continual update scheduling in a simulated evolving corpus. The paper reports consistent improvements over the Layerwise, IPQ, and GenQuery baselines, and includes ablations and scalability analyses in the appendices.

Significance. If the central claim holds, the task is practically valuable for RAG systems and evolving document collections, since it offers a query-free signal for deciding when to retrain an index. The paper's strengths include a broad evaluation across four retrievers and ten BEIR datasets, a released codebase, and several robustness checks: the Appendix F analysis directly shows a monotone relationship between gradient-norm quartiles and d2q recall, and Appendix C shows that the method is stable to the number of sampled in-domain NQ documents and to the number of positives. The claim is nonetheless narrower than the abstract suggests: the validation is confined to BEIR datasets in which document-side and query-side distribution shifts are aligned, so the paper does not yet establish that GradNormIR detects the decoupled document/query shifts that motivate Figure 1. The central derivation is coherent, but several load-bearing definitional and evaluation-protocol issues need to be resolved before the results can be fully trusted.

major comments (5)
  1. [4.2, 5.1] Section 4.2 defines the document-level OOD threshold as 'the median gradient norm of known in-domain documents,' but Section 5.1 states that 'we use the average gradient norm of 3,000 in-domain NQ documents as the reference threshold.' These are different statistics, and the choice directly determines which documents are flagged as OOD in Tables 1 and 3 and in Figure 3. The paper must specify the exact statistic used, and the reproduced experiments must match that definition.
  2. [5.1, Appendix E] The corpus-level threshold gamma=0.5 is introduced in Section 5.1 as 'determined empirically based on preliminary experiments,' and Appendix E clarifies that it was set 'based on the average performance across all datasets.' Because the same ten BEIR datasets are used both to select gamma and to report the corpus-level detection and continual-update results in Section 5.4, the reported behavior is not obtained under a held-out protocol. Please provide a validation procedure that does not use the test corpora (for example, selecting gamma from a separate development split or from in-domain statistics alone) and report how the Section 5.4 update decisions and Figure 3 change with gamma.
  3. [4.2, Figure 1, Limitations] The method uses only document-to-document similarities, and the Limitations paragraph concedes that GradNormIR 'operates solely at the document level without considering potential future queries explicitly.' The motivating example in Figure 1—new 'Willow' quantum-chip documents that are internally coherent but confusable with the existing song corpus—is exactly a decoupled shift: the document-as-query positive pool for such documents is easy, so the gradient norm would be low even though queries will fail. The BEIR corpora used in Table 1 and Appendix F align document and query shifts, so they do not test this scenario. Please add a decoupled-shift experiment (for example, a constructed corpus whose documents form a tight cluster but whose queries are confusable with in-domain terms), or restrict the paper's claim to shifts that are detectable from document-document statistics alone.
  4. [4.2, Appendix F] Pseudo-labels are 'obtained directly from the retriever's own internal similarity scores' (Section 4.2). For a genuinely OOD document with no relevant neighbors in the new corpus, the top-k positive pool is arbitrary, so the gradient norm may measure k-NN instability rather than OOD-ness. The Appendix F correlation between GradNormIR quartiles and d2q recall is computed on BEIR datasets where document-document and query-document similarities are aligned; the paper should either provide evidence on a corpus where the k-NN graph is uninformative for OOD documents, or justify why the signal remains meaningful in that regime.
  5. [5.4, Appendix E] Section 5.4 reports that GradNormIR updates the retriever 'in total N times (N = 6),' while Appendix E states that with gamma=0.5 the OOD corpora are SciFact, Touché, DBPedia, FiQA, Scidocs, NFCorpus, and COVID—seven datasets. This discrepancy is unexplained and directly affects the interpretation of Figure 3, including which sessions trigger updates. In addition, because all selective baselines are forced to the same N=6 updates, the paper should state precisely whether the 'efficiency' claim rests only on 6 updates versus the Naïve baseline's 10 updates.
minor comments (5)
  1. [5.1] The Layerwise baseline is cited as '(Izacard et al., 2022),' but the described layerwise score aggregation across encoder layers is the method of Darrin et al. (2024), which is already cited in the Related Work; the citation should be corrected.
  2. [5, first paragraph] The opening sentence of Section 5 reads 'First, Next, we verify...' and contains a duplicated phrase; this sentence should be rewritten.
  3. [Appendix B] Appendix B begins with 'During the rebuttal period, anonymous reviewers raised...' which is review-process text and should not appear in an archival version of the paper.
  4. [5.1] The corpus C is defined as 'the set of documents with at least one annotated relevant query,' which is a subset of the full BEIR corpus; the potential effect of excluding unlabeled documents on the gradient norms and OOD ratios should be discussed, since the practical pre-indexing setting would include all documents.
  5. [4.2, Eq. (4)] The notation for positives (d+_i from the positive pool) and hard negatives (d^-_ij from D^-(d+_i)) is overloaded and the sampling procedure is described in prose across two paragraphs; a single unambiguous formal definition of the loss used in Eq. (4) would improve reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: the corpus-level retraining trigger uses a threshold fit to the same BEIR evaluation data; document-level GradNormIR is validated externally.

  1. fitted input called prediction [Section 5.1 (hyperparameters) and Appendix E; used in Section 5.4 (Continual Updates)]
    "We set the OOD corpus prediction threshold (γ) to 0.5. We determine this value empirically based on preliminary experiments. ... To predict OOD corpora, we set γ to 0.5 based on the average performance across all datasets. With this threshold, we identify Scifact, Touch’e, DBPedia, FiQA, Scidocs, NFCorpus, and COVID as OOD corpora."

    The binary OOD-corpus labels that drive the continual-update policy are obtained by thresholding r(C) with γ=0.5. The paper states that this threshold was chosen empirically based on the same BEIR datasets where the update experiment is then evaluated, and Appendix E confirms it was set from average performance across all datasets and lists the resulting OOD set. Consequently the retraining schedule (N=6, and which corpora trigger updates in Figure 3) is fitted on the evaluation data, not predicted out-of-sample. The reported advantage of GradNormIR's selective updates is therefore partly determined by the fitted threshold; the 'prediction' of OOD corpora reduces to a decision rule optimized on the test corpora.

full rationale

GradNormIR's document-level score is built from the retriever's own k-NN pseudo-labels (Section 4.2), but it is validated against external query annotations through DRR (Eq. 5) and the d2q analysis in Appendix F; no identity makes the InfoNCE gradient norm equal to retrieval failure, so the document-detection results are not circular by construction. The self-referential nature of the pseudo-labels is a scope limitation that the authors concede ('GradNormIR operates solely at the document level without considering potential future queries explicitly'): it can miss internally coherent but query-confusable corpora such as the Figure 1 Willow example, but that is a correctness/robustness gap, not an input-output equivalence. The substantive circularity is at the corpus level: γ=0.5 is set empirically on the same BEIR datasets used for the continual-update evaluation, and Appendix E confirms it was chosen 'based on the average performance across all datasets.' This makes the binary OOD-corpus trigger and hence the retraining schedule (N=6) in Section 5.4 fitted on the evaluation data, partially determining the reported selective-update advantage. Self-citations (Ko et al. 2024; Kim et al. 2024) appear only as RAG motivation and are not load-bearing. Because Table 1 DRR and Table 2 retriever selection are independent of γ, the circularity is partial, not total.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The method rests on several unproven domain assumptions: gradient norm of a self-supervised contrastive loss indicates generalization failure; dropout perturbation exposes instability; NQ documents define the in-domain reference; and a corpus is OOD if its flagged-document ratio exceeds a threshold tuned on the evaluation benchmark. Free parameters are set by ablation or preliminary experiments. No new physical entities are introduced.

free parameters (7)
  • Number of positive samples p = 8
    Chosen based on ablations; DRR varies with p (Appendix C.2).
  • Number of hard negatives n = 4
    Set to reduce computational cost; described as traditional (Appendix A).
  • Dropout rate for document query = 0.02
    Ablated in Appendix C.2; higher dropout degrades E5.
  • Corpus OOD threshold gamma = 0.5
    Set empirically based on average BEIR performance (Section 5.1, Appendix E); directly controls which corpora trigger updates.
  • Contrastive loss temperature tau = 0.05 (0.01 for E5)
    Ad hoc per retriever; E5 requires 0.01 because its score distribution is skewed (Appendix A).
  • Number of in-domain NQ reference documents = 3000
    Threshold uses the average gradient norm of these documents; ablations use 1000 and 2000 (Section 5.1, Appendix C.1).
  • Positive k-NN pool size k = not reported
    Section 4.2 says 'retrieve the top-k nearest documents' but no value is given in the main text or appendix.
assumptions (5)
  • domain assumption The gradient norm of a contrastive loss is a valid proxy for model uncertainty and generalizability in dense retrieval.
    Adopted from image classification GradNorm and GDScore (Section 4.1); not proven for retrieval models.
  • domain assumption Dropout perturbation of a document query's embedding exposes generalization failure through larger gradient norms.
    Stated as motivation in Section 4.2; the causal link is assumed, not derived.
  • domain assumption Natural Questions documents are representative of the in-domain distribution for all four tested retrievers.
    Section 5.1 says 'since all test retrievers are already trained on NQ'; if a retriever was not trained on NQ, the reference threshold is invalid.
  • ad hoc to paper A corpus is OOD if the proportion of flagged documents exceeds a fixed threshold gamma.
    Defined in Equation (3) with gamma chosen empirically; no theoretical justification is given for 0.5.
  • domain assumption A document is OOD if it is not retrieved in the top-K results for its relevant query.
    Equation (2) defines OOD operationally through retrieval failure, conflating distribution shift with general retrieval difficulty.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR." pith.science (2026). https://pith.science/paper/CC3WETSP

@misc{pith2026250601877,
  author       = {Pith},
  title        = {Pith review of: When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CC3WETSP}},
  note         = {Machine review of arXiv:2506.01877}
}
read the original abstract

Dense retrievers encode texts into embeddings to efficiently retrieve relevant documents from large databases in response to user queries. However, real-world corpora continually evolve, leading to a shift from the original training distribution of the retriever. Without timely updates or retraining, indexing newly emerging documents can degrade retrieval performance for future queries. Thus, identifying when a dense retriever requires an update is critical for maintaining robust retrieval systems. In this paper, we propose a novel task of predicting whether a corpus is out-of-distribution (OOD) relative to a dense retriever before indexing. Addressing this task allows us to proactively manage retriever updates, preventing potential retrieval failures. We introduce GradNormIR, an unsupervised approach that leverages gradient norms to detect OOD corpora effectively. Experiments on the BEIR benchmark demonstrate that GradNormIR enables timely updates of dense retrievers in evolving document collections, significantly enhancing retrieval robustness and efficiency.

Figures

Figures reproduced from arXiv: 2506.01877 by the authors.

Figure 1
Figure 1. Motivation. In evolving corpora, indexing documents that dense retrievers fail to generalize to can severely degrade retrieval performance. Therefore, proactively detecting OOD corpora prior to indexing, without relying on available queries, is essential for maintaining retrieval effectiveness. To address this, we propose GradNormIR, an unsupervised method lever￾aging gradient norms to identify such OOD corpora. are… view at source ↗
Figure 2
Figure 2. Dropout for the document query representa [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Average of Recall@100 across S1 to St with respect to the upper bound for each dataset using a single-trained retriever. Although the trend decreases due to the expanding document corpus over sessions, performance remains robust with continual updates. please refer to Appendix A. Implementations of Baselines. We test three types of baselines: (i) Zero-shot: the retriever re￾mains fixed with no further updates. (ii) … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The prompt template to create pseudo queries using Llama3.1 8B in zero-shot. We prompt it to generate [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Results of relevance gains via OOD document [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Relation Between OOD Ratio and Perfor￾mance [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Feasibility results of GradNormIR for several recent retrievers on the BEIR benchmark. The x-axis [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

20 extracted references · 7 canonical work pages

  1. [1]

    We prompt it to generate a question along with a corresponding answer to ensure the question can be answered

    Provide one {question}{answer}2) DON’T use phrases such as ‘according to the sentence(s)’ in your question.3) DON’T use phrases in the context verbatim.4) An answer should be an entity or entities.5) Ensure the question can be answered without referring back to the document, assuming domain knowledge.6) Ensure the question includes enough context to be un...

    work page

  2. [3]

    arXiv preprint arXiv:2402.03216

    Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216. Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu, and Xiangzhan Yu

    arXiv

  3. [6]

    In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781

    Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781. Pranav Kasela, Gabriella Pasi, Raffaele Perego, and Nicola Tonellotto

    work page 2020

  4. [7]

    In Proceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 13752–13770, Miami, Florida, USA

    Dy- namicER: Resolving emerging mentions to dynamic entities for RAG. In Proceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 13752–13770, Miami, Florida, USA. Association for Computational Linguistics. Dayoon Ko, Jinyoung Kim, Hahyeon Choi, and Gunhee Kim

    work page 2024

  5. [8]

    arXiv preprint arXiv:2409.02685

    Routerretriever: Exploring the benefits of routing over multiple expert embedding models. arXiv preprint arXiv:2409.02685. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, et al

    arXiv

  6. [10]

    arXiv preprint arXiv:2407.06992

    Ro- bust neural information retrieval: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2407.06992. Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gus- tavo Hernández Ábrego, Ji Ma, Vincent Y . Zhao, Yi Luan, Keith B. Hall, Ming-Wei Chang, and Yinfei Yang

    arXiv

  7. [11]

    Preprint, arXiv:2112.07899

    Large dual encoders are generalizable retrievers. Preprint, arXiv:2112.07899. Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. How context affects language mod- els’ factual predictions. In Automated Knowledge Base Construction. Juan Ramos et al

    arXiv

  8. [12]

    arXiv preprint arXiv:2112.07577

    Gpl: Generative pseudo labeling for unsupervised domain adaptation of dense retrieval. arXiv preprint arXiv:2112.07577. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei

    arXiv

Show all 20 references
  1. [14]

    arXiv preprint arXiv:2402.05672

    Multilin- gual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff

    arXiv

  2. [16]

    arXiv preprint arXiv:2401.08909

    Characterising gradients for unsupervised accuracy estimation under distribution shift. arXiv preprint arXiv:2401.08909. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D Manning

    arXiv

  3. [18]

    In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing , pages 1462–

    Coco-dr: Combating dis- tribution shift in zero-shot dense retrieval with con- trastive and distributionally robust learning. In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing , pages 1462–

    2022

  4. [19]

    Additionally, when comparing the cases with and without dropout, the decrease is significantly higher as the number of positives in- creases

    As the number of pos- itives increases, the DRR generally decreases be- cause more gradient norm values make the method more robust. Additionally, when comparing the cases with and without dropout, the decrease is significantly higher as the number of positives in- creases. Th...

    2022

  5. [2013]

    Journal of experimental social psychology , 49(4):764–766

    Detect- ing outliers: Do not use standard deviation around the mean, use absolute deviation around the me- dian. Journal of experimental social psychology , 49(4):764–766. Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanji...

    2023 arXiv

  6. [2016]

    arXiv preprint arXiv:1611.09268

    Ms marco: A human generated ma- chine reading comprehension dataset. arXiv preprint arXiv:1611.09268. Maciej Besta, Ales Kubicek, Roman Niggli, Robert Gerstenberger, Lucas Weitzendorf, Mingyuan Chi, Patrick Iff, Joanna Gajda, Piotr Nyczyk, Jürgen Müller, et al

    arXiv

  7. [2018]

    In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2369–2380

    Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2369–2380. Yue Yu, Chenyan Xiong, Si Sun, Chao Zhang, and Arnold Overwijk

    2018

  8. [2020]

    In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 7870–7881, Online

    Recall and learn: Fine-tuning deep pretrained language models with less forgetting. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 7870–7881, Online. As- sociation for Computational Linguistics. Tao Chen, Mingyang Zha...

    2020

  9. [2021]

    arXiv preprint arXiv:2104.08821

    Simcse: Simple contrastive learning of sentence em- beddings. arXiv preprint arXiv:2104.08821. Yunhao Ge, Yuecheng Li, Di Wu, Ao Xu, Adam M Jones, Amanda Sofie Rios, Iordanis Fostiropoulos, Po-Hsuan Huang, Zachary William Murdock, Gozde Sahin, et al

    arXiv

  10. [2022]

    arXiv preprint arXiv:2212.03533

    Text embeddings by weakly- supervised contrastive pre-training. arXiv preprint arXiv:2212.03533. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei

    arXiv

  11. [2023]

    Preprint, arXiv:2309.07597

    C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.07597. Renchunzi Xie, Ambroise Odonnat, Vasilii Feofanov, Ievgen Redko, Jianfeng Zhang, and Bo An

    arXiv

  12. [2024]

    arXiv preprint arXiv:2406.05085

    Multi-head rag: Solving multi-aspect problems with llms. arXiv preprint arXiv:2406.05085. Yinqiong Cai, Keping Bi, Yixing Fan, Jiafeng Guo, Wei Chen, and Xueqi Cheng

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.