Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Zero-Indexing Internet Search Augmented Generation for Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a zero-indexing pipeline—parser keyword extraction, mixed re-ranking, and a tag-emitting extractor—lets live web search replace a maintained vector index, improving output quality while cutting generative-LLM input tokens…

desk verdict A solid production-systems paper whose headline quality claim is weaker than the evidence: the architecture and extractor benchmarks are worth a look, but the end-to-end comparison leans on a judge that trained the model. read the letter →

arxiv 2411.19478 v2 pith:LKU7FEIT submitted 2024-11-29 cs.IR

Guangxin He , Zonghong Dai , Jiangcheng Zhu , Binqiang Zhao , Qicheng Hu , Chenyue Li , You Peng , Chen Wang
show 1 more author
Binhang Yuan
This is my paper · ORCID
classification cs.IR
keywords retrievalaugmentedgenerationinternetsearchzero-indexingLLMinformationextractiondirectpreferenceoptimizationengineAPIcontent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that live web search can replace a maintained vector index in retrieval-augmented generation for large language models. It proposes a zero-indexing pipeline built from three collaborating components: a small parser LLM that decides in one pass whether internet augmentation is needed and extracts search keywords; a mixed re-ranking strategy that scores both the snippet and the full text of each returned HTML page with an embedding model; and a fine-tuned nine-billion-parameter extractor LLM that returns only the tags of sentences relevant to the user request. On real production queries, the authors report that this pipeline produces responses judged significantly better than those of naive web-context concatenation and of vector-database RAG while feeding the generative LLM 47% and 21% fewer input tokens respectively. The significance is practical: if correct, a deployed conversational AI can stay current with the web—news, prices, recent events—without rebuilding or storing any document index, cutting both staleness and token cost.

What carries the argument

The load-bearing object is the extractor-LLM, a 9B-parameter model that outputs only the TAG-ids of relevant sentences instead of copying text. It is first fine-tuned on instruction pairs where a summary bridges each request to the correct tags, then aligned by direct preference optimization with the judge's choices between responses built from its own extraction and a GPT-4O alternative extraction. Around it sit two supporting mechanisms: the parser-LLM, a 6B model fine-tuned to combine retrieval-determination and keyword extraction in one inference while handling time sensitivity, multilingual queries and out-of-vocabulary terms; and a mixed ranking strategy that treats the search engine's snippet and the page's full content as separate candidates, doubles the pool from n to 2n, and re-ranks them with the BGE-M3 embedding model to offset engagement-driven ranking bias.

What would settle it

Take the same 463 real production queries and have the responses from internet-SAG-ext and each baseline re-judged by human annotators, or by a capable judge model that played no role in training data generation or extraction; if human preference does not reproduce the WIN-minus-LOSE margins, or if the roughly 40-45% of pairs currently discarded as position-inconsistent turn out to be the decisive ones, the quality claim collapses even though the architecture may still be sound.

Watch

Extended reading notes

Core claim

The central claim is that a content-condensing extractor trained specifically for search results is what makes internet-augmented generation viable in production. The paper shows that naively concatenating retrieved HTML text degrades generative quality because excessive sequence length and irrelevant content interfere with the model, whereas the extractor-LLM, trained by supervised fine-tuning followed by direct preference optimization, selects only the relevant sentences and emits them as lightweight tag identifiers. This filtering lets a cheaper prompt succeed: outputs are judged higher quality than both concatenation and top-k chunk retrieval across three generative backbones (two open models around 34B-72B and one larger closed model), with the token-cost reduction following from the fact that only extracted text enters the generative prompt.

Load-bearing premise

The headline 'significantly improved quality' rests on the assumption that GPT-4O's pairwise preference, after discarding position-inconsistent pairs, is an unbiased measure of response quality—yet the same model shaped the extractor's DPO training and served as the alternative extractor, so the judge is entangled with the system it is evaluating.

Editorial extensions

If this is right

  • Production RAG services can drop offline corpus maintenance and query live search APIs instead, eliminating stale documents and index-storage overhead.
  • Generative-LLM token cost falls by 21% versus vector-database retrieval and 47% versus naive concatenation on the measured requests, directly lowering per-query API cost.
  • The extractor-LLM rejects irrelevant content with an Exact-Match rate above 0.79 on real requests, so the final prompt contains less noise for any downstream generative model.
  • Because the parser and extractor handle retrieval and filtering, the generative backbone itself needs no retraining, so the pipeline applies to any newer LLM as it is released.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 21% token saving holds at larger scale, the tag-emitting extractor design suggests a general recipe: train a small model to output span identifiers, not text, for any long-context filtering stage.
  • The snippet-plus-full-content re-ranking is a portable trick: combining a search engine's own summary with the page body as two separate candidates could reduce ranking bias in any web-scraping pipeline, not just RAG.
  • The reported WIN/TIE/LOSE counts include a large P-BIA fraction (roughly a quarter to nearly half of comparisons); if those discarded pairs are not random, a human-preference audit could narrow the claimed quality gap.
  • One testable extension would be mixing results from Google and Bing or different regional engines and re-ranking the union, since the mixed ranking already assumes the engine order is unreliable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a zero-indexing Internet search augmented generation system for LLMs. A Parser-LLM decides in a single inference whether web search is needed and extracts search keywords; a mixed ranking strategy reranks search-engine results using BGE-M3 over both snippets and full HTML content; and an Extractor-LLM, trained with SFT and DPO, selects relevant tagged sentences from fetched pages. The system is evaluated end-to-end against naive Internet-SAG and VectorDB-RAG with three generative LLMs, using GPT-4o pairwise judgments (Table 1) and input-token cost (Table 2), and the extractor is evaluated on synthetic, open-source, and real-world benchmarks (Tables 3-5). The authors report improved response quality at lower generative-LLM input-token cost and state that the system is deployed at 01.AI.

Significance. If the empirical claims hold, the paper would make a useful practical contribution: live web search could substitute for a maintained document index in production RAG, with lower input-token costs. The paper has concrete strengths: it describes a deployed system, gives a detailed training-data construction pipeline for the parser and extractor, includes robustness-oriented extractor benchmarks, and reports token-cost reductions of 21% versus VectorDB-RAG and 47% versus naive Internet-SAG. The central weakness is that the headline quality claim rests on GPT-4o pairwise judgments, while GPT-4o also generated training labels, served as the alternative extractor in DPO data construction, and judged the DPO preferences. That evaluation loop, combined with the large fraction of discarded position-biased comparisons and the absence of significance tests, means the claim of significantly improved quality is not yet established.

major comments (4)
  1. [Section 5.1, Table 1] The claim of significantly improved quality is not supported by the reported numbers alone. For five of the six rows, 44-47% of the 463 comparisons are discarded as P-BIA (209, 216, 219, 219, and 205 cases), and the sixth row discards 25%. The paper assumes these are pure noise, but provides no sensitivity analysis, no confidence intervals, and no significance test. If P-BIA cases were treated as ties or as missing data with different assumptions, the win/loss ratios would change materially. The anecdotal explanations for QWEN2-72B's poor VectorDB-RAG results, such as the model creating its own questions, are also not systematically documented. Please report statistical tests, error bars, and an analysis of the P-BIA cases.
  2. [Sections 3.1, 4.2, 4.4, and 5.1] The evaluation judge is not independent of the systems being judged. GPT-4o generated the Parser-LLM instruction labels (Section 3.1), generated the SFT summaries for the Extractor-LLM (Section 4.2), acted as the alternative extractor whose outputs define the DPO preference pairs (Section 4.4), judged those DPO preferences, and then judged the end-to-end comparisons in Table 1. Since the Extractor-LLM is explicitly trained to produce outputs that GPT-4o scores highly, the end-to-end quality advantage may partly reflect a self-preference artifact. A concrete remedy is to re-evaluate a subsample with human annotators, or at least with a different judge model that was not used anywhere in the training pipeline, and report inter-judge agreement.
  3. [Section 5.2, Open-source Benchmark] The extractor benchmark is partially circular. For the multi-answer tasks, ground truth for document-dependent questions is generated by GPT-4o, and for multi-question tasks, GPT-4o refines the answer sentences to produce ground-truth tags. Table 4 then compares the Extractor-LLM against GPT-4o on this same GPT-4o-derived ground truth. This does not invalidate the benchmark, but it weakens the comparison, especially since the Extractor-LLM was trained with GPT-4o preferences. Please add human-verified ground truth for these subsets, or report the agreement between GPT-4o-generated labels and human labels.
  4. [Section 5.1, Baseline Systems] The end-to-end comparison includes only naive Internet-SAG and VectorDB-RAG, yet the paper claims superiority over 'other RAG paradigms' and cites prior search-augmented systems such as WebGPT, RQ-RAG, DRAGIN, and RankRAG in Related Work. None of these is compared empirically. At minimum, the claims should be narrowed to the two implemented baselines, or at least one strong search-augmented or query-rewriting baseline should be included; otherwise the stated claim is broader than the evidence.
minor comments (5)
  1. [Throughout] There are numerous typos and formatting issues, including 'update-to-date', 'GERERATIVE-LLM', 'Sythetic', 'Reponse', and 'mininal'. A careful proofreading pass is needed.
  2. [Section 3.2] The mixed ranking strategy is described only qualitatively. The paper does not specify how snippet-level and full-content BGE-M3 scores are normalized or combined, how the top-K distinct HTML files are selected when snippet and full-content scores disagree, or how results from auxiliary keyword lists are merged. An algorithm box or a small worked example would make this component reproducible.
  3. [Section 4.4] The DPO data preparation repeats Step 2 and Step 3 only twice per request, with no stated random seed, no justification for the choice of two repeats, and no reporting of the total number of DPO pairs after filtering. Please clarify the dataset size and the selection procedure for the randomly chosen tagged content.
  4. [Section 4.2] The instruction set is said to keep the None-ratio at 5% 'determined based on empirical tests', but those tests are not reported. A sensitivity analysis of this ratio, or at least a citation to a prior study, would support this design choice.
  5. [Related Work, Reference [8]] Reference [8] (Dotori) is a key-value SSD storage paper and appears to be mis-cited in the list of Internet search augmented generation works. Please verify the citation placement.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline quality claim is partially circular: the extractor is DPO-trained against GPT-4o preferences, and the end-to-end quality advantage is then measured by the same GPT-4o judge.

  1. fitted input called prediction [Section 4.4 (DPO data preparation) and Section 5.1 (Evaluation Metrics, Table 1)]
    "In the second DPO phase, we further optimize the EXTRACTOR -LLM ’s output TAGs towards an end-to-end preference estimated by OpenAI API (i.e.,GPT-4 O) considered as aLLM judge. ... We ask the LLM judge to score each response separately based on how well it answers theRequest. ... We employGPT-4 O as the judge to evaluate the quality of responses generated by different RAG systems."

    The DPO phase constructs y+ and y- from GPT-4o's pairwise response scores, so the extractor is trained to produce tags that make GPT-4o prefer the final generated response. Table 1 then reports as 'significantly improved quality' the pairwise preference of the same GPT-4o judge between the trained system and baselines that received no such judge-alignment training. The headline quality advantage is therefore partly the DPO objective re-measured by its own reward model; the qualitative claim is statistically forced by construction. The token-cost reduction in Table 2 is independent, but the 'significantly improved quality' conclusion is not.

  2. fitted input called prediction [Section 5.2 (Open-source benchmark construction, Tables 4-5) and Section 4.2 (SFT instruction construction)]
    "Document-dependent questions with auxiliary information of answer numbers are provided to GPT-4o to generate the ground truth. ... we utilize GPT-4o to refine these answers, extracting only the target sentences while ignoring irrelevant parts, thereby providing more accurate and fine-grained ground truth tags for our multi-question tasks."

    The EXTRACTOR-LLM extraction-accuracy results are scored against ground-truth tags generated by GPT-4o, the same API family that produced the SFT instruction labels in Section 4.2 ('we need to use LLMs to process the input data ... we ask the LLM not only to generate the desired collection of TAGs'). Thus the benchmark measures proximity to the training-signal generator's labeling style, not an independent ground truth; the reported high scores for EXTRACTOR-LLM are in part agreement with its own teacher signal.

full rationale

This is not an equation-level tautology, and the system has independent engineering content: the zero-indexing architecture, parser-LLM, mixed ranking, and measured 21%/47% token-cost reductions stand on their own. However, the central qualitative claim that Internet-SAG-Ext generates 'significantly improved quality' rests on Table 1, where GPT-4o is both the training-time reward signal for the DPO-trained extractor and the evaluation-time judge. Because DPO explicitly optimizes extraction toward GPT-4o's pairwise response preferences, and the evaluation uses that same preference model, part of the reported WIN advantage is circular. The evaluation is further fragile because 25-47% of comparisons per row are discarded as P-BIA without sensitivity analysis, and no significance tests are reported, but that fragility is a validity concern rather than circularity. Overall, the paper makes genuine non-circular contributions, but its headline quality claim is partially circular due to the judge-alignment loop, so a score of 6 is appropriate.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces software components (Parser-LLM, Extractor-LLM, mixed ranking) rather than postulated physical or theoretical entities; these are trained artifacts, not invented entities lacking evidence. The main ledger entries are the empirical assumptions: the GPT-4o judge as a quality oracle, BGE-M3 as a bias corrector, GPT-4o-generated labels as training signal, and the production sample as representative. These assumptions are load-bearing for the headline quality and cost claims.

free parameters (3)
  • None ratio in SFT instruction set = 5%
    Section 4.2 sets the share of instructions expected to return None to 5%, 'determined based on empirical tests'; this tunes the extractor's rejection behavior and directly affects precision and exact-match scores.
  • VectorDB-RAG baseline hyperparameters = chunk-size 512, chunk-overlap 128, top-k 12, neighbor-num 1
    Section 5.1 fixes these values for the baseline; they determine how much irrelevant context the baseline sees and therefore influence the size of the reported quality gap.
  • DPO preference repeats per request = twice
    Section 4.4 repeats Step 2 and Step 3 only twice per request to include more requests; this manually chosen budget shapes the DPO dataset diversity.
assumptions (4)
  • domain assumption GPT-4o pairwise judgments, after position-swap consistency filtering, are a valid and unbiased measure of response quality.
    Section 5.1 uses GPT-4o as the sole end-to-end judge and discards inconsistent pairs as P-BIA; the 'significantly improved quality' claim rests on this.
  • domain assumption BGE-M3 relevance scoring on snippets and full HTML content can correct search engine ranking bias.
    Section 3.2 assumes engagement-driven ranking bias is mitigated by independent semantic relevance scores.
  • domain assumption GPT-4o-generated instruction labels, summaries, and DPO preference labels are accurate enough to train the parser and extractor.
    Sections 3.1, 4.2, and 4.4 use GPT-4o outputs as ground truth or preference signals without human verification beyond a manual None-ratio check.
  • domain assumption The 463 cleaned real-world queries are representative of general production traffic and independent of the instruction-set data.
    Section 5.1 states no overlap after deduplication, but gives no distributional similarity analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Zero-Indexing Internet Search Augmented Generation for Large Language Models." pith.science (2026). https://pith.science/paper/LKU7FEIT

@misc{pith2026241119478,
  author       = {Pith},
  title        = {Pith review of: Zero-Indexing Internet Search Augmented Generation for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LKU7FEIT}},
  note         = {Machine review of arXiv:2411.19478}
}
read the original abstract

Retrieval augmented generation has emerged as an effective method to enhance large language model performance. This approach typically relies on an internal retrieval module that uses various indexing mechanisms to manage a static pre-processed corpus. However, such a paradigm often falls short when it is necessary to integrate the most up-to-date information that has not been updated into the corpus during generative inference time. In this paper, we explore an alternative approach that leverages standard search engine APIs to dynamically integrate the latest online information (without maintaining any index for any fixed corpus), thereby improving the quality of generated content. We design a collaborative LLM-based paradigm, where we include: (i) a parser-LLM that determines if the Internet augmented generation is demanded and extracts the search keywords if so with a single inference; (ii) a mixed ranking strategy that re-ranks the retrieved HTML files to eliminate bias introduced from the search engine API; and (iii) an extractor-LLM that can accurately and efficiently extract relevant information from the fresh content in each HTML file. We conduct extensive empirical studies to evaluate the performance of this Internet search augmented generation paradigm. The experimental results demonstrate that our method generates content with significantly improved quality. Our system has been successfully deployed in a production environment to serve 01.AI's generative inference requests.

Figures

Figures reproduced from arXiv: 2411.19478 by the authors.

Figure 1
Figure 1. A comparison between the standard RAG paradigm and our Internet search augmented generation [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An illustration about pre-processing the retrieved HTML file and Extractor LLM processing procedure. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. An illustration Of DPO Training Preparation. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Characterizing Web Search in The Age of Generative AI

    cs.IR 2025-10 conditional novelty 6.0 of 10

    AI search engines vary greatly in how much they rely on web pages versus internal model knowledge, and these differences shift which sources and concepts users see.

Reference graph

Works this paper leans on

62 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  2. [2]

    Retrieval-augmented generation for ai-generated content: A survey.arXiv preprint arXiv:2402.19473, 2024

    Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. Retrieval-augmented generation for ai-generated content: A survey.arXiv preprint arXiv:2402.19473, 2024

    arXiv 2024

  3. [3]

    A survey on rag meets llms: Towards retrieval-augmented large language models

    Yujuan Ding, Wenqi Fan, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meets llms: Towards retrieval-augmented large language models. arXiv preprint arXiv:2405.06211, 2024

    arXiv 2024

  4. [4]

    Webgpt: Browser-assisted question-answering with human feedback

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021

    arXiv 2021

  5. [5]

    Internet-augmented dialogue generation

    Mojtaba Komeili, Kurt Shuster, and Jason Weston. Internet-augmented dialogue generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8460–8478, 2022

    work page 2022

  6. [6]

    Internet-augmented language models through few-shot prompting for open-domain question answering

    Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. Internet-augmented language models through few-shot prompting for open-domain question answering. arXiv preprint arXiv:2203.05115, 2022

    arXiv 2022

  7. [7]

    The web can be your oyster for improving large language models.arXiv preprint arXiv:2305.10998, 2023

    Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jingyuan Wang, Jian-Yun Nie, and Ji-Rong Wen. The web can be your oyster for improving large language models.arXiv preprint arXiv:2305.10998, 2023

    arXiv 2023

  8. [8]

    Dotori: A key-value ssd based kv store

    Carl Duffy, Jaehoon Shim, Sang-Hoon Kim, and Jin-Soo Kim. Dotori: A key-value ssd based kv store. Proceedings of the VLDB Endowment, 16(6):1560–1572, 2023

    work page 2023

Show all 62 references
  1. [9]

    Weknow-rag: An adaptive approach for retrieval-augmented generation integrating web search and knowledge graphs

    Weijian Xie, Xuefeng Liang, Yuhui Liu, Kaihua Ni, Hong Cheng, and Zetian Hu. Weknow-rag: An adaptive approach for retrieval-augmented generation integrating web search and knowledge graphs. arXiv preprint arXiv:2408.07611, 2024

    2024 arXiv

  2. [10]

    Introducing chatgpt search

    OpenAI. Introducing chatgpt search. https://openai.com/index/introducing-chatgpt-search/, 2024

    2024

  3. [11]

    Term-weighting approaches in automatic text retrieval

    Gerard Salton and Christopher Buckley. Term-weighting approaches in automatic text retrieval. Information processing & management, 24(5):513–523, 1988

    1988

  4. [12]

    The probabilistic relevance framework: Bm25 and beyond

    Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389, 2009

    2009

  5. [13]

    Sparkly: A simple yet surprisingly strong tf/idf blocker for entity matching

    Derek Paulsen, Yash Govind, and AnHai Doan. Sparkly: A simple yet surprisingly strong tf/idf blocker for entity matching. Proceedings of the VLDB Endowment, 16(6):1507–1519, 2023

    2023

  6. [14]

    Multilingual e5 text embeddings: A technical report.arXiv preprint arXiv:2402.05672, 2024

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilingual e5 text embeddings: A technical report.arXiv preprint arXiv:2402.05672, 2024. 18

    2024 arXiv

  7. [15]

    M3-embedding: Multi- linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M3-embedding: Multi- linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics ACL 2024, page...

    2024

  8. [16]

    Nv-embed: Improved techniques for training llms as generalist embedding models.arXiv preprint arXiv:2405.17428, 2024

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nv-embed: Improved techniques for training llms as generalist embedding models.arXiv preprint arXiv:2405.17428, 2024

    2024 arXiv

  9. [17]

    Lider: an efficient high-dimensional learned index for large-scale dense passage retrieval

    Yifan Wang, Haodi Ma, and Daisy Zhe Wang. Lider: an efficient high-dimensional learned index for large-scale dense passage retrieval. Proceedings of the VLDB Endowment, 16(2):154–166, 2022

    2022

  10. [18]

    Plin: A persistent learned index for non-volatile memory with high performance and instant recovery .Proceedings of the VLDB Endowment, 16(2):243–255, 2022

    Zhou Zhang, Zhaole Chu, Peiquan Jin, Yongping Luo, Xike Xie, Shouhong Wan, Yun Luo, Xufei Wu, Peng Zou, Chunyang Zheng, et al. Plin: A persistent learned index for non-volatile memory with high performance and instant recovery .Proceedings of the VLDB Endowment, 16(2):243–255, 2022

    2022

  11. [19]

    Towards efficient index construction and approximate nearest neighbor search in high-dimensional spaces.Proceedings of the VLDB Endowment, 16(8):1979–1991, 2023

    Xi Zhao, Yao Tian, Kai Huang, Bolong Zheng, and Xiaofang Zhou. Towards efficient index construction and approximate nearest neighbor search in high-dimensional spaces.Proceedings of the VLDB Endowment, 16(8):1979–1991, 2023

    1979

  12. [20]

    Learned index: A comprehensive experimental evaluation

    Zhaoyan Sun, Xuanhe Zhou, and Guoliang Li. Learned index: A comprehensive experimental evaluation. Proceedings of the VLDB Endowment, 16(8):1992–2004, 2023

    1992

  13. [21]

    Dili: A distribution-driven learned index

    Pengfei Li, Hua Lu, Rong Zhu, Bolin Ding, Long Yang, and Gang Pan. Dili: A distribution-driven learned index. Proceedings of the VLDB Endowment, 16(9):2212–2224, 2023

    2023

  14. [22]

    Pre-trained embeddings for entity resolution: an experimental analysis

    Alexandros Zeakis, George Papadakis, Dimitrios Skoutas, and Manolis Koubarakis. Pre-trained embeddings for entity resolution: an experimental analysis. Proceedings of the VLDB Endowment , 16(9):2225–2238, 2023

    2023

  15. [23]

    Adaptive in- dexing in high-dimensional metric spaces.Proceedings of the VLDB Endowment, 16(10):2525–2537, 2023

    Konstantinos Lampropoulos, Fatemeh Zardbani, Nikos Mamoulis, and Panagiotis Karras. Adaptive in- dexing in high-dimensional metric spaces.Proceedings of the VLDB Endowment, 16(10):2525–2537, 2023

    2023

  16. [24]

    Hyper: A high-performance and memory-efficient learned index via hybrid construction.Proceedings of the ACM on Management of Data, 2(3):1–26, 2024

    Shunkang Zhang, Ji Qi, Xin Yao, and Andr´e Brinkmann. Hyper: A high-performance and memory-efficient learned index via hybrid construction.Proceedings of the ACM on Management of Data, 2(3):1–26, 2024

    2024

  17. [25]

    Bonsaikv: Towards fast, scalable, and persistent key-value stores with tiered, heterogeneous memory system

    Miao Cai, Junru Shen, Yifan Yuan, Zhihao Qu, and Baoliu Ye. Bonsaikv: Towards fast, scalable, and persistent key-value stores with tiered, heterogeneous memory system. Proceedings of the VLDB Endowment, 17(4):726–739, 2023

    2023

  18. [26]

    Experimental analysis of large-scale learnable vector storage compression

    Hailin Zhang, Penghao Zhao, Xupeng Miao, Yingxia Shao, Zirui Liu, Tong Yang, and Bin Cui. Experimental analysis of large-scale learnable vector storage compression. Proceedings of the VLDB Endowment , 17(4):808–822, 2023

    2023

  19. [27]

    Fluidkv: Seamlessly bridging the gap between indexing performance and memory-footprint on ultra-fast storage.Proceedings of the VLDB Endowment, 17(6):1377–1390, 2024

    Ziyi Lu, Qiang Cao, Hong Jiang, Yuxing Chen, Jie Yao, and Anqun Pan. Fluidkv: Seamlessly bridging the gap between indexing performance and memory-footprint on ultra-fast storage.Proceedings of the VLDB Endowment, 17(6):1377–1390, 2024

    2024

  20. [28]

    Treeline: an update-in-place key-value store for modern storage

    Geoffrey X Yu, Markos Markakis, Andreas Kipf, Per-˚Ake Larson, Umar Farooq Minhas, and Tim Kraska. Treeline: an update-in-place key-value store for modern storage. Proceedings of the VLDB Endowment, 16(1), 2022

    2022

  21. [29]

    Pantheon: Private retrieval from public key-value store

    Ishtiyaque Ahmad, Divyakant Agrawal, Amr El Abbadi, and Trinabh Gupta. Pantheon: Private retrieval from public key-value store. Proceedings of the VLDB Endowment, 16(4):643–656, 2022

    2022

  22. [30]

    Mirrorkv: An efficient key-value store on hybrid cloud storage with balanced performance of compaction and querying.Proceedings of the ACM on Management of Data, 1(4):1–27, 2023

    Zhiqi Wang and Zili Shao. Mirrorkv: An efficient key-value store on hybrid cloud storage with balanced performance of compaction and querying.Proceedings of the ACM on Management of Data, 1(4):1–27, 2023. 19

    2023

  23. [31]

    Learning to optimize lsm-trees: Towards a reinforcement learning based key-value store for dynamic workloads

    Dingheng Mo, Fanchao Chen, Siqiang Luo, and Caihua Shan. Learning to optimize lsm-trees: Towards a reinforcement learning based key-value store for dynamic workloads. Proceedings of the ACM on Management of Data, 1(3):1–25, 2023

    2023

  24. [32]

    Vexless: A serverless vector data management system using cloud functions

    Yongye Su, Yinqi Sun, Minjia Zhang, and Jianguo Wang. Vexless: A serverless vector data management system using cloud functions. Proceedings of the ACM on Management of Data, 2(3):1–26, 2024

    2024

  25. [33]

    Openai gpt-4o, 2024

    OpenAI. Openai gpt-4o, 2024

    2024

  26. [34]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Abhimanyu Dubey , Abhinav Jauhri, Abhinav Pandey , Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  27. [35]

    Yi: Open foundation models by 01

    Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jian- qun Chen, Jing Chang, et al. Yi: Open foundation models by 01. ai.arXiv preprint arXiv:2403.04652, 2024

    2024 arXiv

  28. [36]

    The claude 3 model family: Opus, sonnet, haiku, 2024

    Anthropic. The claude 3 model family: Opus, sonnet, haiku, 2024

    2024

  29. [37]

    Ruler: What’s the real context size of your long-context language models? arXiv preprint arXiv:2404.06654, 2024

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. Ruler: What’s the real context size of your long-context language models? arXiv preprint arXiv:2404.06654, 2024

    2024 arXiv

  30. [38]

    Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human preference

    Zachary Levonian, Chenglu Li, Wangda Zhu, Anoushka Gade, Owen Henkel, Millie-Ellen Postle, and Wanli Xing. Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human preference. arXiv preprint arXiv:2310.03184, 2023

    arXiv

  31. [39]

    Query rewriting in retrieval-augmented large language models

    Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303–5315, 2023

    2023

  32. [40]

    Dragin: Dynamic retrieval augmented generation based on the real-time information needs of large language models

    Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. Dragin: Dynamic retrieval augmented generation based on the real-time information needs of large language models. arXiv preprint arXiv:2403.10081, 2024

    2024 arXiv

  33. [41]

    Rq-rag: Learning to refine queries for retrieval augmented generation.arXiv preprint arXiv:2404.00610, 2024

    Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu. Rq-rag: Learning to refine queries for retrieval augmented generation.arXiv preprint arXiv:2404.00610, 2024

    2024 arXiv

  34. [42]

    Predicting clicks: estimating the click-through rate for new ads

    Matthew Richardson, Ewa Dominowska, and Robert Ragno. Predicting clicks: estimating the click-through rate for new ads. InProceedings of the 16th international conference on World Wide Web, pages 521–530, 2007

    2007

  35. [43]

    Beyond clicks: dwell time for personalization

    Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. Beyond clicks: dwell time for personalization. In Proceedings of the 8th ACM Conference on Recommender systems, pages 113–120, 2014

    2014

  36. [44]

    A database of ambiguous chinese characters with measures for meaning dominance and meaning balance.Applied Psycholinguistics, 45(4):695–716, 2024

    Huilin Chen, Xu Xu, Haiquan Li, Xinyue Yu, Ruting Pan, and Zhaoyang Zhang. A database of ambiguous chinese characters with measures for meaning dominance and meaning balance.Applied Psycholinguistics, 45(4):695–716, 2024

    2024

  37. [45]

    Extreme speed and scale for dl training and inference

    Microsoft. Extreme speed and scale for dl training and inference. https://www.deepspeed.ai/, 2024

    2024

  38. [46]

    Bge m3-embedding: Multi-lingual, multi-functionality , multi-granularity text embeddings through self-knowledge distillation

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multi-lingual, multi-functionality , multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216, 2024

    2024 arXiv

  39. [47]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 20

    2024

  40. [48]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024

    2024

  41. [49]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model, 2024

    2024

  42. [50]

    The faiss library

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazar´e, Maria Lomeli, Lucas Hosseini, and Herv´e J´egou. The faiss library . 2024

    2024

  43. [51]

    Llmtest needle in a haystack - pressure testing llms

    gkamradt. Llmtest needle in a haystack - pressure testing llms. https://github.com/gkamradt/ LLMTest_NeedleInAHaystack, 2023

    2023

  44. [52]

    Needlebench: Can llms do retrieval and reasoning in 1 million context window?, 2024

    Mo Li, Songyang Zhang, Yunxin Liu, and Kai Chen. Needlebench: Can llms do retrieval and reasoning in 1 million context window?, 2024

    2024

  45. [53]

    Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps.arXiv preprint arXiv:2011.01060, 2020

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps.arXiv preprint arXiv:2011.01060, 2020

    2011 arXiv

  46. [54]

    Multi-task inference: Can large language models follow multiple instructions at once?arXiv preprint arXiv:2402.11597, 2024

    Guijin Son, Sangwon Baek, Sangdae Nam, Ilgyun Jeong, and Seungone Kim. Multi-task inference: Can large language models follow multiple instructions at once?arXiv preprint arXiv:2402.11597, 2024

    2024 arXiv

  47. [55]

    Large language models can be lazy learners: Analyze shortcuts in in-context learning

    Ruixiang Tang, Dehan Kong, Longtao Huang, and Hui Xue. Large language models can be lazy learners: Analyze shortcuts in in-context learning. InFindings of the Association for Computational Linguistics: ACL 2023, pages 4645–4657, 2023

    2023

  48. [56]

    Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Donia Scott, Nuria Bel, and Chengqing Zong, editors, Proceedings of the 28th International Conference on Computational Lingu...

    2020

  49. [57]

    https://github.com/multipledata/MTQA, 2023

    Mtqa. https://github.com/multipledata/MTQA, 2023

    2023

  50. [58]

    Multi-task inference: Can large language models follow multiple instructions at once? 2024

    Guijin Son, Sangwon Baek, Sangdae Nam, Ilgyun Jeong, and Seungone Kim. Multi-task inference: Can large language models follow multiple instructions at once? 2024

    2024

  51. [59]

    Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering.Transactions of the Association for Computation...

    2023

  52. [60]

    Search augmented instruction learning

    Hongyin Luo, Tianhua Zhang, Yung-Sung Chuang, Yuan Gong, Yoon Kim, Xixin Wu, Helen Meng, and James Glass. Search augmented instruction learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3717–3729, 2023

    2023

  53. [61]

    Making retrieval-augmented language models robust to irrelevant context

    Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. Making retrieval-augmented language models robust to irrelevant context. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  54. [62]

    Rankrag: Unifying context ranking with retrieval-augmented generation in llms

    Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. arXiv preprint arXiv:2407.02485, 2024. 21

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.