Pith. sign in

REVIEW 4 major objections 6 minor 66 references

MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a fully on-device RAG pipeline can cut search latency, memory, and power use while holding accuracy by pairing a cluster-partitioned vector index with selective pruning of retrieved text.

desk verdict Real on-device measurements make this a useful systems contribution; the accuracy-maintenance claim and the analytical models need tightening, but the paper deserves serious refereeing. read the letter →

arxiv 2507.01079 v1 pith:YFACQ57Y submitted 2025-07-01 cs.DB

Taehwan Park , Geonho Lee , Min-Soo Kim This is my paper
classification cs.DB
keywords retrieval-augmentedgenerationon-deviceRAGapproximatenearestneighborsearchgraph-basedvectorindexselectivecontentreductionmobilelargelanguagemodelsmemory-efficientindexingpower-efficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MobileRAG sets out to make retrieval-augmented generation practical on a phone, where RAM, battery, and heat budgets are tight. It combines EcoVector, a vector index that keeps a small graph of cluster centers in memory and loads per-cluster graphs from disk only when needed, with Selective Content Reduction (SCR), which re-chunks retrieved documents and keeps only the sentences most similar to the query before sending them to a small language model. The paper reports that on a Galaxy S24 this beats standard vector-search and RAG pipelines by 1.72–8.89x in search latency, reduces time to first token by 1.18–1.41x, cuts memory by 10.7–54.5%, and cuts power by 24.4–40.2%, with accuracy matching or exceeding baselines. The payoff, if true, is that private personal data can be searched, summarized, and questioned entirely on-device.

What carries the argument

The two load-bearing mechanisms are EcoVector and Selective Content Reduction. EcoVector is a two-level graph index: a small HNSW graph (a hierarchical navigable small-world graph) over cluster centroids lives in RAM, while each cluster's own HNSW graph is stored on disk and loaded cluster-by-cluster during a query, so distance computations—the dominant power cost—are cut while memory stays near disk-based IVF. SCR is a post-retrieval filter: it splits each retrieved document into overlapping sentence windows, scores every window against the query with the same embedding model used for retrieval, keeps the top window plus a fixed number of neighboring sentences, merges the kept text across documents, and reorders documents by their best window score so that the prompt fed to the small language model is both shorter and better ordered.

What would settle it

Take the three QA benchmarks used in the paper and, for each query with a known gold answer, record whether the SCR-selected window actually contains the gold answer sentence. If that containment rate is far below the reported accuracy, or if answer accuracy on the excluded sentences is near chance, then the claim that SCR maintains accuracy is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the two perceived bottlenecks of on-device RAG—the vector index and the language-model input—can both be attacked without surrendering accuracy, by restructuring where computation happens. EcoVector partitions embeddings into clusters, builds a hierarchical navigable small-world graph over the few thousand centroids that stays in RAM, and stores separate small graphs for each cluster's vectors on flash storage, loading and unloading them per query. This trades some disk I/O for far fewer CPU distance computations, which the paper argues is a net win because CPU draw dominates mobile power. SCR then re-embeds sliding windows of the retrieved documents, keeps the single most query-similar window per document plus a small context extension, merges the survivors, and reorders documents by their top window score. Together the two components shrink both the search cost and the token count handed to the small language model; the experiments claim accuracy is maintained or improved while latency, memory, and energy all drop.

Load-bearing premise

The claim that SCR keeps accuracy rests on sentence-level embedding similarity being a reliable proxy for the sentences the language model actually needs, yet the paper never measures how often the selected window contains the gold answer.

Editorial extensions

If this is right

  • On the reported hardware, EcoVector reaches 0.93 recall@10 on a million-vector SIFT set with 1.72–8.89x lower search latency than IVF, HNSW, and their disk-based variants.
  • SCR trims average per-document input tokens by 42% on SQuAD, 31% on TriviaQA, and 7% on HotpotQA at unchanged accuracy, which translates into 10.4–41.6% lower time-to-first-token depending on the small language model.
  • Across the full RAG pipeline the method lowers power by 24.4–40.2% and memory by 10.7–54.5% compared with Naive-RAG, EdgeRAG, and Advanced RAG, with the gains growing for larger on-device models.
  • Because the index, retrieval, and generation all run locally, a phone can answer queries about personal documents with no network round trip, which is the privacy argument the paper makes.
  • SCR's document reordering step behaves like a reranker without a separate model, letting MobileRAG roughly match Advanced RAG accuracy without the extra inference cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not measure how often SCR's top-1 window actually contains the gold answer; an obvious extension is an answer-aware or trained selector that could make the accuracy guarantee robust rather than benchmark-tuned.
  • EcoVector's two-level design suggests a natural hierarchy for very large personal corpora: cluster again within each disk-resident cluster to keep per-query I/O bounded as the collection grows past what one level of partitioning handles.
  • The power argument depends on phone flash being fast enough that disk reads are cheap relative to CPU cycles; on phones with slower eMMC storage the balance could reverse, so the reported gains may not transfer to the lowest-end devices.
  • The reported SCR parameters (window size 3, overlap 2, context extension 1) are fixed across all experiments and tuned on the evaluation benchmarks; a parameter-free or per-query adaptive window size is a testable way to tell whether the method's accuracy hold depends on that tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper introduces MobileRAG, a fully on-device RAG pipeline that combines EcoVector, a disk-partitioned graph vector index, with Selective Content Reduction (SCR), a post-retrieval method that scores sentence windows against the query embedding and keeps only the most relevant content. The authors derive analytical memory, latency, and power models, and report experiments on a Galaxy S24 using SIFT and NYTimes for ANNS and SQuAD, HotpotQA, and TriviaQA with three small language models. The central claims are 1.72–8.89x search latency improvements, 10.7–54.5% memory reductions, 24.4–40.2% power reductions, and accuracy maintained relative to Naive-RAG and other baselines.

Significance. The problem is timely and the evaluation is genuinely on-device, with multiple datasets and small language models, which is a strength. If the claims hold, the EcoVector design of keeping a small centroid HNSW graph in RAM while paging per-cluster graphs from disk is a practical recipe for mobile vector search, and the SCR token-reduction idea would be useful for on-device RAG. However, three load-bearing points currently need additional support: the power model rests on current draws that are not independently sourced, the SCR accuracy-preservation claim is not verified by an answer-containment analysis and its hyperparameters are selected on the test benchmarks, and the EcoVector memory expression appears to undercount the loaded-cluster graph size. These issues do not invalidate the system concept, but they materially affect the strength of the central claims.

major comments (4)
  1. [Section 3.4.3; Figures 9 and 13] The power model uses I(t_s)=2300 μA and I(t_d)=800 μA, described only as "determined" with citations to general smartphone power analyses rather than to a measurement protocol on the Galaxy S24/Exynos 2400. Because these two constants enter the theoretical power values in Figure 9 and the retrieval-power discussion in Figure 13, the power advantage is partially fitted to the same experiments it is used to explain. Please provide an independent source or a direct measurement protocol for these current draws, and validate predicted vs. measured energy on a held-out configuration.
  2. [Section 4, Steps 1–2; Section 5.3.1; Table 5] SCR keeps only the top-1 sliding window per retrieved document based on GTE-Small cosine similarity to the query. The paper never reports how often the retained window contains the gold answer span, or both supporting sentences for HotpotQA, so the "maintaining accuracy" claim could partly reflect the small LM answering from parametric memory rather than from the preserved evidence. In addition, the SCR hyperparameters (sliding_window_size, overlap_size, context_extension_size) are selected by inspecting Figure 12 on the same three test benchmarks that are later used for Table 5; this is tuning on the test set, not a blind test of the sentence-similarity proxy. Please add an answer-containment analysis and a proper train/validation/test split for SCR hyperparameter selection.
  3. [Table 1, EcoVector row] The loaded-cluster memory term 4(d + M'/(1-p0)) has the units of a single graph node, not a cluster. Since each inverted-list graph contains N/N_c vectors, this term should be (N/N_c)·4(d + M'/(1-p0)) (or an equivalent justification should be given). As written, the expression understates EcoVector's RAM footprint by roughly a factor of N/N_c in the graph term, which affects the theoretical memory curves in Figure 6 and the memory-reduction percentages stated in the abstract and conclusion.
  4. [Section 3.4.2 and Table 2] The text defines T_search = t_s + t_d, but Table 2 lists only CPU-side search-time expressions such as ef_c·M' + n_P·ef_L·M' for EcoVector, with no corresponding disk-I/O term. Since the analytic power values in Section 3.4.3 require t_d, the end-to-end search-time expression should be stated explicitly and used consistently in the theoretical comparisons; otherwise the analytic latency and power contributions are incomplete.
minor comments (6)
  1. [Section 5.3] The accuracy metric is not defined; please state whether it is exact match, F1, or another measure, and describe the evaluation protocol for the three QA datasets.
  2. [Section 4, Steps 1–2] The description alternates between scoring "sliding windows" and scoring "chunks" (e.g., the example assigns scores to Chunk1–Chunk5), which makes it unclear whether the window score is the max, mean, or some other aggregate of sentence scores. Please align the terminology and define the scoring precisely.
  3. [Figure 13 caption] The caption reads "N1, N0.9, N0.7, N0.6 denote Naive-RAG at chunk ratios 1, 0.9, 0.8, 0.6," which mismatches N0.7 and the ratio 0.8. Please correct this inconsistency.
  4. [Figures 6 and 9] The figures mix actual measured values and "Theoretical Value" markers without a clear legend; please distinguish measured bars from analytically computed values so the reader can see how well the models predict the measurements.
  5. [Table 5] No variance or number of runs is reported. Please add standard deviations or confidence intervals, or at least state that each number is a single run, so the magnitude of the reported differences can be assessed.
  6. [Section 7, Conclusion] The summary line "improving search latency by 1.72–8.89 times (at 0.93 recall@10 for SIFT)" should specify the dataset, recall level, and whether this is retrieval-only or end-to-end latency, because the abstract's latency claim is broader than the evidence shown.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EcoVector/SCR claims rest on standard analytical models and empirical measurements, not on reductions to their own inputs.

full rationale

The derivation chain for MobileRAG is not circular. EcoVector's memory expressions in Table 1 are standard accounting of centroids, inverted lists, and graph links; the search-latency expressions in Table 2 use externally cited UFS 4.0 specifications and CPU-cycle counts. The power estimate in Section 3.4.3 combines these times with two current-draw constants, I(t_s)≈2300 µA and I(t_d)≈800 µA, citing references [1,5]. Although the text says 'we have determined' without showing the determination, there is no statement that these constants were fitted to the Figure 9 measurements, so no fitted-parameter-renamed-as-prediction step can be exhibited. SCR's accuracy claim is empirical rather than definitional: the method selects top-scoring windows using GTE-Small similarity, and Table 5 reports the resulting accuracy. The choice of sliding_window_size=3, overlap_size=2, and context_extension_size=1 from Figure 12 on the same QA benchmarks is a test-set tuning concern that limits generalization claims, but it does not make the reported accuracy equivalent to the selection rule by construction. No load-bearing self-citations appear in the reference list. Therefore no circular step meets the evidentiary bar set by the analysis rules.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several fitted or assumed quantities. The SCR component has three hyperparameters tuned on the same test benchmarks, and the power model depends on two current-draw values that appear calibrated to the test device. The analytical memory and latency models rely on standard HNSW approximations plus device-specific storage and current assumptions. No new physical entities are introduced. The two domain assumptions about k-means cluster balance and sentence-embedding relevance are the most fragile: if either fails, the corresponding component (EcoVector or SCR) loses its stated benefit.

free parameters (7)
  • sliding_window_size = 3
    Selected from Figure 12 sweep over window sizes on SQuAD/HotpotQA/TriviaQA test sets; no held-out validation.
  • overlap_size = 2
    Selected in the same sweep; with context_extension_size=1, produces overlapping 5-sentence windows.
  • context_extension_size = 1
    Selected in the same sweep; appends one sentence before and after the chosen window to preserve context.
  • number of clusters (N_c) for EcoVector = not stated
    N_c directly affects memory, latency, and power (Figure 11), but the value used in the main results is not reported.
  • HNSW parameters (M, efConstruction, efSearch) = not stated
    Graph construction and search width parameters are not specified for EcoVector or the baselines, making the exact operating points unreproducible.
  • CPU current draw I(t_s) = 2300 uA
    Used in the power model (Section 3.4.3); 'determined' without an independent source, likely calibrated to the same Galaxy S24 experiments.
  • disk I/O current draw I(t_d) = 800 uA
    Same as above; the ratio of CPU to I/O current drives the power advantage claim.
assumptions (5)
  • standard math HNSW graph traversal with expansion factor ef returns accurate k-ANNS results in about ef*M operations per level.
    Used in Section 3.4.2 latency formulas for HNSW, HNSWPQ, IVF-HNSW, and EcoVector; a standard approximation for HNSW behavior.
  • domain assumption k-means clustering produces clusters that are balanced enough that each inverted-list graph contains roughly N/N_c points and is small enough to load partially on a mobile device.
    Section 3.1.1 and Figure 8a; if clusters are highly imbalanced, some loaded clusters would be large, increasing memory and latency and possibly violating the partial-loading design.
  • domain assumption UFS 4.0 specifications (T_seek=0.025ms, T_cmd=0.015ms, T_transfer=3.6e-7 ms/Byte) accurately describe the Galaxy S24's storage behavior.
    Section 3.4.2 uses these values to model disk I/O latency; real device behavior, including OS caching and thermal throttling, may differ.
  • ad hoc to paper CPU distance computations consume roughly 3x more current than disk I/O on this phone (I(t_s)=2300uA vs I(t_d)=800uA).
    Section 3.4.3 states these values as determined, with no source or measurement protocol; this assumption is what makes EcoVector's power model favor disk I/O over CPU work.
  • domain assumption GTE-Small sentence embeddings are a reliable proxy for which sentences the language model needs to answer the query correctly.
    Section 4 Step 1 scores each sliding window against the query using the embedding model; if this similarity does not track answer usefulness, SCR will remove relevant content and degrade accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG." pith.science (2026). https://pith.science/paper/YFACQ57Y

@misc{pith2026250701079,
  author       = {Pith},
  title        = {Pith review of: MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFACQ57Y}},
  note         = {Machine review of arXiv:2507.01079}
}
read the original abstract

Retrieval-Augmented Generation (RAG) has proven effective on server infrastructures, but its application on mobile devices is still underexplored due to limited memory and power resources. Existing vector search and RAG solutions largely assume abundant computation resources, making them impractical for on-device scenarios. In this paper, we propose MobileRAG, a fully on-device pipeline that overcomes these limitations by combining a mobile-friendly vector search algorithm, \textit{EcoVector}, with a lightweight \textit{Selective Content Reduction} (SCR) method. By partitioning and partially loading index data, EcoVector drastically reduces both memory footprint and CPU usage, while the SCR method filters out irrelevant text to diminish Language Model (LM) input size without degrading accuracy. Extensive experiments demonstrated that MobileRAG significantly outperforms conventional vector search and RAG methods in terms of latency, memory usage, and power consumption, while maintaining accuracy and enabling offline operation to safeguard privacy in resource-constrained environments.

Figures

Figures reproduced from arXiv: 2507.01079 by the authors.

Figure 1
Figure 1. Comparison RAG Methods with MobileRAG. reducing RAM usage, power consumption, and search la￾tency. Second, we propose the SCR method, which re-chunks retrieved documents, recalculates their similarity scores, and reconstructs the prompt by selecting only the most similar chunks, thereby significantly lowering inference time and energy use. Both components run on-device—eliminating network dependency and preserving p… view at source ↗
Figure 2
Figure 2. Overview of MobileRAG: on-device pipeline consisting of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. EcoVector Architecture. 3.1 Build Method The EcoVector index is constructed in four main stages as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (10 more)
Figure 3
Figure 3. Figure 3: Document References in Chat Application. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 5
Figure 5. Figure 5: Overview of the SCR Method. Step 1: Similarity Computation: Instead of sending full initially retrieved documents directly into the sLM, each doc￾ument is first split into individual sentences. Subsequently, sliding windows of a fixed size sliding_window_size (e.g., th…
Figure 7
Figure 7. Figure 7: Comparison of Recall and QPS. 5.2.2 Analysis for Search Latency [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: a explains how EcoVector exploits multiple small graphs instead of a single massive structure (as in one-graph HNSW). Consequently, Figure 8b indicates that these meth￾ods achieve high recall at a much smaller efSearch width than HNSW. This smaller search width effecti…
Figure 9
Figure 9. Figure 9: Power Consumption. 5.2.4 Analysis for Update Latency [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 11
Figure 11. Figure 11: Memory Usage, Search Latency, and Power Consumption on Various Cluster Sizes. 0 15 30 45 60 100 92 81 71 58 22 0 10 20 30 100 93 82 66 55 43 0 10 20 30 40 50 100 91 83 69 52 27 (a) SQuAD Accuracy (%) Document Share (%) (b) HotpotQA Document Share (%) (c) TriviaQA Docu…
Figure 12
Figure 12. Figure 12: Comparison of SCR-based MobileRAG and Naive-RAG with Small Chunks or Compressor. 5.3 Evaluation of the SCR Method For both queries and documents, we employ the GTE-Small embedding model, which contains approximately 33 mil￾lion parameters [32]. Additionally, we utiliz…
Figure 10
Figure 10. Figure 10: Insertion and Deletion Latency. 5.2.5 Analysis for varying numbers of centroids [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 13
Figure 13. Figure 13: Memory and Power for retrieval: 𝑁1, 𝑁0.9, 𝑁0.7, 𝑁0.6 denote Naive-RAG at chunk ratios 1, 0.9, 0.8, 0.6; 𝐸 denotes EdgeRAG; 𝑀 denotes MobileRAG. Qwen-2.5 1.5B; and Naive-RAG and EdgeRAG by 17.7–40.5%, and AdvancedRAG by 18.5–41.6% in Deepseek-r1 1.5B. These improvement…
Figure 14
Figure 14. Figure 14: provides additional details on TTFT; while the SCR method itself adds a small overhead, the resultant de￾crease in sLM inference time ultimately reduces the total TTFT. Naïve-RAG Edge RAG Advanced RAG MobileRAG 0.36 0.36 0.36 0.36 0.3 0.3 0.5 0.3 0.38 1.32 9.8 12.5 12…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 27 canonical work pages

  1. [1]

    Smartphone SoC Power Consumption and Performance Anal- ysis

    2021. Smartphone SoC Power Consumption and Performance Anal- ysis. https://www.anandtech.com/show/16463/snapdragon-888-vs- exynos-2100-galaxy-s21-ultra/5

    work page 2021

  2. [2]

    Martin Aumüller, Erik Bernhardsson, and Alexander Faithfull. 2020. ANN-Benchmarks: A benchmarking tool for approximate nearest neighbor algorithms. Information Systems 87 (2020), 101374

    work page 2020

  3. [3]

    Artem Babenko and Victor Lempitsky. 2014. The inverted multi-index. IEEE transactions on pattern analysis and machine intelligence 37, 6 (2014), 1247–1260

    2014

  4. [4]

    Qingqing Cao, Noah Weber, Niranjan Balasubramanian, and Aruna Balasubramanian. 2019. DeQA: On-Device Question Answering. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services (Seoul, Republic of Korea) (MobiSys ’19). Association for Computing Machinery, New York, NY, USA, 27–40. doi:10.1145/3307334.3326071

    arXiv 2019

  5. [5]

    Aaron Carroll and Gernot Heiser. 2010. An analysis of power consump- tion in a smartphone. In 2010 USENIX Annual Technical Conference (USENIX ATC 10)

    work page 2010

  6. [6]

    Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. 2021. Spann: Highly- efficient billion-scale approximate nearest neighborhood search. Ad- vances in Neural Information Processing Systems 34 (2021), 5199–5212

    work page 2021

  7. [7]

    Yongjian Chen, Tao Guan, and Cheng Wang. 2010. Approximate nearest neighbor search by residual vector quantization. Sensors 10, 12 (2010), 11259–11273

    work page 2010

  8. [8]

    Wei Dong, Charikar Moses, and Kai Li. 2011. Efficient k-nearest neigh- bor graph construction for generic similarity measures. In Proceedings of the 20th international conference on World wide web . 577–586

    work page 2011

Show all 66 references
  1. [9]

    Tianyu Fan, Jingyuan Wang, Xubin Ren, and Chao Huang. 2025. Mini- RAG: Towards Extremely Simple Retrieval-Augmented Generation. arXiv preprint arXiv:2501.06713 (2025)

    2025 arXiv

  2. [10]

    Cong Fu and Deng Cai. 2016. Efanna: An extremely fast approximate nearest neighbor search algorithm based on knn graph. arXiv preprint arXiv:1609.07228 (2016)

    2016 arXiv

  3. [11]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023. Retrieval- augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 2 (2023)

    2023 arXiv

  4. [12]

    Yunfan Gao, Yun Xiong, Meng Wang, and Haofen Wang. 2024. Modular rag: Transforming rag systems into lego-like reconfigurable frame- works. arXiv preprint arXiv:2407.21059 (2024)

    2024 arXiv

  5. [13]

    Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013. Optimized product quantization for approximate nearest neighbor search. In Proceedings of the IEEE conference on computer vision and pattern recog- nition. 2946–2953

    2013

  6. [14]

    Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Rajaram Naik, Pengshan Cai, and Alfio Gliozzo. 2022. Re2G: Retrieve, rerank, generate. arXiv preprint arXiv:2207.06300 (2022)

    2022 arXiv

  7. [15]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforce- ment learning. arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  8. [16]

    ZIRUI GUO, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. 2024. LightRAG: Simple and Fast Retrieval-Augmented Generation. https: //openreview.net/forum?id=bbVH40jy7f

    2024

  9. [17]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning . PMLR, 3929–3938

    2020

  10. [18]

    Rossi, Subhabrata Mukherjee, Xianfeng Tang, Qi He, Zhigang Hua, Bo Long, Tong Zhao, Neil Shah, Amin Javari, Yinglong Xia, and Jiliang Tang

    Haoyu Han, Yu Wang, Harry Shomer, Kai Guo, Jiayuan Ding, Yongjia Lei, Mahantesh Halappanavar, Ryan A. Rossi, Subhabrata Mukherjee, Xianfeng Tang, Qi He, Zhigang Hua, Bo Long, Tong Zhao, Neil Shah, Amin Javari, Yinglong Xia, and Jiliang Tang

  11. [19]

    Taeho Hwang, Soyeong Jeong, Sukmin Cho, SeungYoon Han, and Jong C Park. 2024. DSLR: Document refinement with sentence-level re-ranking and reconstruction to enhance retrieval-augmented gener- ation. arXiv preprint arXiv:2407.03627 (2024)

    2024 arXiv

  12. [20]

    Piotr Indyk and Rajeev Motwani. 1998. Approximate nearest neigh- bors: towards removing the curse of dimensionality. In Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing (Dallas, Texas, USA) (STOC ’98). Association for Computing Machinery, New York, NY, ...

    1998

  13. [21]

    Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnawamy, and Rohan Kadekodi. 2019. Diskann: Fast accurate billion-point nearest neighbor search on a single node. Advances in Neural Information Processing Systems 32 (2019)

    2019

  14. [22]

    Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence 33, 1 (2010), 117–128

    2010

  15. [23]

    Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2019. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351 (2019)

    2019 arXiv

  16. [24]

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer

  17. [25]

    Herve Jégou, Matthijs Douze, and Cordelia Schmid. 2011. Prod- uct Quantization for Nearest Neighbor Search. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 1 (2011), 117–128. doi:10.1109/TPAMI.2010.57

    2011 doi

  18. [26]

    Yannis Kalantidis and Yannis Avrithis. 2014. Locally optimized product quantization for approximate nearest neighbor search. In Proceedings of the IEEE conference on computer vision and pattern recognition . 2321– 2328

    2014

  19. [27]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering.. In EMNLP (1). 6769–6781

    2020

  20. [28]

    Yoshiyuki Kawano and Keiji Yanai. 2014. ILSVRC on a Smartphone. Information and Media Technologies 9, 3 (2014), 371–375

    2014

  21. [29]

    Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2019. Generalization through memorization: Nearest neighbor language models. arXiv preprint arXiv:1911.00172 (2019)

    2019 arXiv

  22. [30]

    Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. arXiv preprint arXiv:1906.00300 (2019)

    2019 arXiv

  23. [31]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval- Augmented Generation for Knowledge-Intensive NLP Tasks. In Ad- vances ...

    2020

  24. [32]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi- stage contrastive learning. arXiv preprint arXiv:2308.03281 (2023)

    2023 arXiv

  25. [33]

    Jun Liu, Zhenhua Zhu, Jingbo Hu, Hanbo Sun, Li Liu, Lingzhi Liu, Guohao Dai, Huazhong Yang, and Yu Wang. 2022. Optimizing Graph- based Approximate Nearest Neighbor Search: Stronger and Smarter. In 2022 23rd IEEE International Conference on Mobile Data Management (MDM). IEEE, 1...

    2022

  26. [34]

    Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. arXiv preprint arXiv:1908.08345 (2019)

    2019 arXiv

  27. [35]

    Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuan- dong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, et al. 2024. Mobilellm: Optimizing sub-billion parameter language models for on-device use cases. In Forty-first Internat...

    2024

  28. [36]

    Edson Ramiro Lucas Filho, Lambros Odysseos, Yang Lun, Fu Kebo, and Herodotos Herodotou. 2022. DITIS: A Distributed Tiered Storage Simulator. Infocommunications Journal 14, 4 (2022)

    2022

  29. [37]

    Yury Malkov, Alexander Ponomarenko, Andrey Logvinov, and Vladimir Krylov. 2014. Approximate nearest neighbor algorithm based on navigable small world graphs. Information Systems 45 (2014), 61– 68

    2014

  30. [38]

    Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence 42, 4 (2018), 824–836

    2018

  31. [39]

    Malkov and D

    Yu A. Malkov and D. A. Yashunin. 2020. Efficient and Robust Approx- imate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 4 (2020), 824–836. doi:10.1109/TPAMI.2018.2889473

    2020

  32. [40]

    Ross McGowan, Jinru Su, Vince DiCocco, Thejaswi Muniyappa, and Grant Strimel. 2021. SmallER: Scaling neural entity resolution for edge devices. (2021). https://www.amazon.science/publications/smaller- scaling-neural-entity-resolution-for-edge-devices

    2021

  33. [41]

    David Nister and Henrik Stewenius. 2006. Scalable recognition with a vocabulary tree. In2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06) , Vol. 2. Ieee, 2161–2168

    2006

  34. [42]

    Ashwin Paranjape, Omar Khattab, Christopher Potts, Matei Zaharia, and Christopher D Manning. 2021. Hindsight: Posterior-guided train- ing of retrievers for improved open-ended generation. arXiv preprint arXiv:2110.07752 (2021)

    2021 arXiv

  35. [43]

    Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yaz- dani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, et al. 2020. KILT: a benchmark for knowledge intensive language tasks. arXiv preprint arXiv:2009.02252 (2020)

    2020 arXiv

  36. [44]

    Liudmila Prokhorenkova and Aleksandr Shekhovtsov. 2020. Graph- based nearest neighbor search: From practice to theory. InInternational Conference on Machine Learning . PMLR, 7803–7813

    2020

  37. [45]

    Guanqiao Qu, Qiyuan Chen, Wei Wei, Zheng Lin, Xianhao Chen, and Kaibin Huang. 2025. Mobile edge intelligence for large language models: A contemporary survey. IEEE Communications Surveys & Tutorials (2025)

    2025

  38. [46]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang

  39. [47]

    V Sanh. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

    2019 arXiv

  40. [48]

    Korakit Seemakhupt, Sihang Liu, and Samira Khan. 2024. EdgeRAG: Online-Indexed RAG for Edge Devices.arXiv preprint arXiv:2412.21023 (2024)

    2024 arXiv

  41. [49]

    Samsung Semiconductor. [n. d.]. UFS 4.0 Performance and Power Efficiency Metrics. https://semiconductor.samsung.com/news- events/tech-blog/samsung-develops-first-ufs-4-0-storage-solution- compliant-with-new-industry-standard/#:~:text=will%20deliver% 20approximately%202x%20and,...

  42. [50]

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652 (2023)

    2023 arXiv

  43. [51]

    Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020. Mobilebert: a compact task-agnostic bert for resource-limited devices. arXiv preprint arXiv:2004.02984 (2020)

    2020 arXiv

  44. [52]

    Google SRE Team. [n. d.]. Latency Figures for Memory and Storage in Modern Processors. https://static.googleusercontent.com/media/sre. google/ko//static/pdf/rule-of-thumb-latency-numbers-letter.pdf#:~: text=L1%20cache%20reference%201%20Branch,010

  45. [53]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleaving retrieval with chain-of-thought rea- soning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509 (2022)

    2022 arXiv

  46. [54]

    Mengzhao Wang, Xiaoliang Xu, Qiang Yue, and Yuxiang Wang. 2021. A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search. arXiv preprint arXiv:2101.12631 (2021)

    2021 arXiv

  47. [55]

    Yile Wang, Peng Li, Maosong Sun, and Yang Liu. 2023. Self-knowledge guided retrieval augmentation for large language models. arXiv preprint arXiv:2310.05002 (2023)

    2023 arXiv

  48. [56]

    Wang and Duen Horng Chau

    Zijie J. Wang and Duen Horng Chau. 2024. MeMemo: On-device Re- trieval Augmentation for Private and Personalized Text Generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SI- GIR ’2...

    2024

  49. [57]

    Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, and Jimmy Lin. 2020. DeeBERT: Dynamic early exiting for accelerating BERT inference. arXiv preprint arXiv:2004.12993 (2020)

    2020 arXiv

  50. [58]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  51. [59]

    Hao Yang, Min Zhang, and Daimeng Wei. 2024. IRAG: Iterative Re- trieval Augmented Generation for SLU. In2024 20th IEEE International Colloquium on Signal Processing & Its Applications (CSPA). IEEE, 30–34

    2024

  52. [60]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hot- potQA: A dataset for diverse, explainable multi-hop question answer- ing. arXiv preprint arXiv:1809.09600 (2018)

    2018 arXiv

  53. [61]

    Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. Ad- vances in Neural Information Processing Systems 37 (2024), 121156– 121184

    2024

  54. [62]

    Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2025. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. Ad- vances in Neural Information Processing Systems 37 (2025), 121156– 121184

    2025

  55. [63]

    Ting Zhang, Chao Du, and Jingdong Wang. 2014. Composite quan- tization for approximate nearest neighbor search. In International Conference on Machine Learning . PMLR, 838–846. 14

    2014

  56. [2016]

    arXiv preprint arXiv:1606.05250 (2016)

    Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)

    2016 arXiv

  57. [2017]

    arXiv preprint arXiv:1705.03551 (2017)

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551 (2017)

    2017 arXiv

  58. [2025]

    arXiv:2501.00309 [cs.IR] https://arxiv.org/abs/2501.00309

    Retrieval-Augmented Generation with Graphs (GraphRAG). arXiv:2501.00309 [cs.IR] https://arxiv.org/abs/2501.00309

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.