REVIEW 4 major objections 5 minor 45 references
WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read WebANNS claims in-browser vector search over multi-gigabyte data can hit 10ms latency with up to 743.8x P99 speedup and 39% lower memory.
desk verdict A genuinely fast in-browser ANNS system, but the headline speedup lacks a recall measurement, so the 743.8x claim is not yet an ANNS claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is a three-tier data hierarchy: WebAssembly holds a small hot cache of vectors and the HNSW graph and performs all distance and sort operations at near-native speed; a JavaScript tier acts as an intermediate cache, data-exchange bridge, and adapter between WebAssembly's synchronous execution and IndexedDB's asynchronous API; IndexedDB holds the full dataset on disk. Two further mechanisms carry the performance claims: a phased lazy-loading algorithm that accumulates vectors missing from memory into a list and loads them only when the list exceeds the HNSW candidate size $ef$ or when moving to the next graph layer, preserving correct entry points while issuing few disk transactions; and a heuristic memory optimizer that models disk accesses under random and optimal fetching, then shrinks the cache along an interpolated latency curve until measured access counts exceed a threshold $\theta$.
What would settle it
Run both engines on the same query set and dataset, fix recall@10 (or any stated recall target) to be equal, and compare P99 latency and memory. If WebANNS's advantage shrinks or disappears once recall is matched, the claimed 743.8x improvement is largely a quality trade-off; if the gap persists at equal recall, the claim stands.
Extended reading notes
Core claim
The paper's discovery, on its own terms, is that browser-imposed constraints on ANNS are not separate walls but coupled design levers: moving distance computations and sort operations into WebAssembly frees the computational bottleneck, phased lazy loading eliminates the disk bottleneck while preserving HNSW's greedy path, and treating query latency as a black-box function of cache size lets the engine find a near-minimal memory footprint automatically. The paper is trying to show that with these three changes, an HNSW-based in-browser engine can handle datasets that crash the previous state-of-the-art engine, returns queries in the 10ms range on a 7.5GB corpus, and cuts memory by up to 39% while retaining sub-second latency even at a 20% memory-data ratio.
Load-bearing premise
The headline speedup assumes WebANNS and the baseline engine return search results of comparable quality; the paper reports latency and memory but no recall or accuracy measure, so part of the reported gain could reflect a less thorough search rather than a fundamentally faster engine.
Editorial extensions
If this is right
- In-browser retrieval-augmented generation over private, personalized data becomes feasible at scale: a 7.5GB dataset can be searched in tens of milliseconds instead of tens of seconds, so sensitive data need not leave the device.
- WebAssembly alone accounts for a 2-5.33x speedup on small in-memory datasets; the larger gains come from lazy loading and adaptive caching.
- Browsers using HNSW can adopt the phased lazy-loading rule: defer disk loads until the ignored list exceeds the candidate-list size, avoiding redundant IndexedDB accesses while preserving correct search paths.
- The adaptive memory optimizer can shrink the engine's footprint by 7-39% depending on browser and device while keeping P99 latency within a user-set threshold.
- The supported dataset size in a browser expands by at least 8.5x compared with the baseline engine, since WebANNS no longer crashes on datasets above roughly 900MB.
Reading between the lines
- Beyond the paper's reported metrics, the 743.8x figure should be read as a ceiling until recall is measured; the authors report latency and memory only, so a matched-quality comparison is the natural next test.
- The black-box memory optimizer could generalize beyond ANNS: any web workload combining an in-memory cache with IndexedDB could use the same latency-threshold shrink loop, since the optimizer treats the query path as an opaque function of cache size.
- The phased lazy-loading rule—defer disk loads until the ignored list exceeds the candidate size or the layer ends—looks transferable to other graph traversals that must interleave synchronous WebAssembly computation with asynchronous browser I/O.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents WebANNS, an in-browser approximate nearest neighbor search engine built on HNSW, with three main optimizations: WebAssembly-based computation to reduce CPU overhead, a three-tier cache hierarchy (Wasm, JavaScript, IndexedDB) with phased lazy loading to reduce external storage accesses, and a heuristic cache-size optimizer that iteratively reduces memory while keeping query latency under user-set thresholds. The system is evaluated on five datasets, three browsers, and three devices, reporting P99 query latency and memory usage, with claims of up to 743.8x P99 speedup over the state-of-the-art in-browser engine Mememo and 7-39% memory savings. The paper also contains analytical models for random and optimal fetching behavior and an ablation study separating the effects of Wasm, lazy loading, and cache optimization.
Significance. If the latency and memory claims are correct at equal search quality, the contribution is practically significant: it would make multi-gigabyte, privacy-preserving ANNS feasible in a web browser at 10-millisecond query latencies, which matters for in-browser RAG and personalized web applications. Strengths of the paper include a clear system design, a reproducible prototype with a public repository, an ablation study (Table 2), and an evaluation across heterogeneous browsers and devices. The central weakness is that the speedup claims are not accompanied by any recall or accuracy metric, so the reported speedups may partly reflect an accuracy-latency tradeoff rather than a strictly better engine. In addition, the analytical derivation behind the memory optimizer contains a mathematical error that should be corrected before the claims can be fully trusted.
major comments (4)
- [Section 4.2, Tables 1-2, Algorithm 1] No recall, precision, or result-overlap metric is reported anywhere in the paper; Section 4.2 defines only query latency and memory usage. This is load-bearing because Algorithm 1 (lines 14-16 and 24-31) defers out-of-memory neighbors and evaluates them later against a candidate list W whose distance threshold may have tightened, so a vector that would have been explored in an in-memory HNSW search can be excluded. The 743.8x P99 improvement in Table 1 and the ablation gains in Table 2 could therefore partly reflect fewer or less accurate results. Please report recall@10 or result-overlap@10 for both WebANNS and Mememo on the same query sets, and state the HNSW construction and search parameters (M, efConstruction, ef) used by both engines.
- [Section 3.4, Equation (3)] Equation (3) does not match the proof that precedes it. Under the stated random-fetching assumptions, with N items, |Q| distinct queried vectors, and memory capacity M, the expected number of disk accesses is ndb = 1 + (|Q|-1)*(N-M)/(N-1), which equals |Q| at M=1. The printed formula gives |Q|+1 at M=1 and differs from the proof's expression for general M. Since Algorithm 2 extrapolates from the random-fetching curve to choose the next memory size C_next, this is not a purely cosmetic typo; either the equation or the proof should be corrected and the curve-fitting step re-examined.
- [Section 3.4, Algorithm 2, line 14] The formula k = (n_Q - ndb) / (1 - C_test) is dimensionally inconsistent with the rest of the section. Equation (3) and Figure 6 are expressed in terms of nmem and a normalized memory axis, while C_test is described as a memory size in megabytes and reported in megabytes in Table 3. As written, the algorithm does not define how MB is converted to vector counts or to the normalized x-axis of Figure 6. Please specify the exact normalization and state whether C_0, C_i, and theta are memory ratios, vector counts, or byte counts.
- [Abstract, Tables 1 and 3] The headline numbers in the abstract are not obtained under a single experimental condition. The 743.8x speedup in Table 1 is for Wiki-60k on Mac/Chrome, and the Wiki-480k row has N/A for Mememo, so no direct comparison supports a speedup on the 7.5GB dataset. The 'up to 39%' memory reduction in Table 3 comes from a different dataset (Wiki-50k) under the cache optimizer with p=0.8 and T_theta=100ms, where P99 query latency is 151-276ms rather than the 10ms range reported in Table 1. The paper should state precisely which dataset, browser, and configuration each abstract claim refers to.
minor comments (5)
- [Table 1] The 'Win/Firefox' row appears twice with identical values; one of the duplicate rows should be removed.
- [Table 2] The row structure is difficult to read because the Mememo, WebANNS-Base, and WebANNS rows are not explicitly labeled in the table body; adding row labels or a legend would improve clarity.
- [Algorithm 1] The condition 'should be a candidate' (lines 17 and 27) is never formally defined; please state the condition explicitly (for example, |W| < ef or distance(e,q) < distance(f,q)).
- [Section 4.3] The sentence 'WebANNS can complete the retrieval of the 7.5GB Wiki-480k dataset in 72ms' should specify the device and browser, since Table 1 reports Wiki-480k latencies ranging from 23.46ms to 72.00ms depending on the environment.
- [Section 4.4] The phrase 'WebANNS-Base, with Wasm and three-tier caching, archives at least an order-of-magnitude improvement' contains a typo: 'archives' should be 'achieves'.
Circularity Check
No significant circularity: the headline latency and memory claims are measured against an external baseline (Mememo), and the Section 3.4 models are derived from stated fetching assumptions with user-set thresholds, not fitted to the reported results.
full rationale
WebANNS's central claims are comparative latency and memory measurements against Mememo (Tables 1-3), not quantities derived from its own assumptions. The analytic model in Section 3.4 (Equations 2-4) is derived from explicit random-fetching and optimal-fetching assumptions, each with a stated proof, and the heuristic optimizer (Algorithm 2) uses actual QUERY_TEST measurements with user-set thresholds p=0.8 and T_theta=100ms to choose a memory size; the reported savings are the result of that optimization, not a prediction forced by the model. Algorithm 1's phased lazy loading is an implementation strategy, and while its correctness is asserted rather than proven, that is an omitted-proof or overclaim concern, not circularity. The only self-citations (e.g., [20], [42], [43], which share co-author Xuanzhe Liu) are contextual related-work references and do not carry any load-bearing argument. The absence of recall@k or other accuracy metrics is a substantive evaluation gap, since the 743.8x latency improvement may partly reflect an accuracy-latency tradeoff, but that is a correctness and completeness concern rather than a circularity. No load-bearing step reduces to its inputs by definition or by self-citation, so the paper is self-contained against its external baseline.
Assumptions & free parameters
free parameters (4)
- HNSW search parameter ef =
not reported
- HNSW construction parameters M and efConstruction =
not reported
- memory optimizer percentage threshold p =
0.8
- memory optimizer absolute time threshold T_theta =
100ms
assumptions (5)
- domain assumption HNSW graph-based ANNS is an appropriate index for in-browser use
- domain assumption WebAssembly is faster than JavaScript for this workload
- domain assumption IndexedDB access is a dominant cost and all-in-one loading is faster
- ad hoc to paper Random-fetching model: each disk access loads the queried vector plus M-1 random vectors, memory initially empty
- ad hoc to paper The real fetching curve lies between the random and optimal curves
Cite this review
Pith. "Pith review of WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers." pith.science (2026). https://pith.science/paper/ZLMYFJOK
@misc{pith2026250700521,
author = {Pith},
title = {Pith review of: WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZLMYFJOK}},
note = {Machine review of arXiv:2507.00521}
}
abstract
Approximate nearest neighbor search (ANNS) has become vital to modern AI infrastructure, particularly in retrieval-augmented generation (RAG) applications. Numerous in-browser ANNS engines have emerged to seamlessly integrate with popular LLM-based web applications, while addressing privacy protection and challenges of heterogeneous device deployments. However, web browsers present unique challenges for ANNS, including computational limitations, external storage access issues, and memory utilization constraints, which state-of-the-art (SOTA) solutions fail to address comprehensively. We propose WebANNS, a novel ANNS engine specifically designed for web browsers. WebANNS leverages WebAssembly to overcome computational bottlenecks, designs a lazy loading strategy to optimize data retrieval from external storage, and applies a heuristic approach to reduce memory usage. Experiments show that WebANNS is fast and memory efficient, achieving up to $743.8\times$ improvement in 99th percentile query latency over the SOTA engine, while reducing memory usage by up to 39\%. Note that WebANNS decreases query time from 10 seconds to the 10-millisecond range in browsers, making in-browser ANNS practical with user-acceptable latency.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Total canvas memory use exceeds the maximum limit (Safari 12)
2018. Total canvas memory use exceeds the maximum limit (Safari 12). https://stackoverflow.com/questions/52532614/total-canvas-memory-use- exceeds-the-maximum-limit-safari-12. Accessed: 2025-01-01
-
[2]
Wikipedia (en) embedded with cohere.ai multilingual-22-12 encoder
2024. Wikipedia (en) embedded with cohere.ai multilingual-22-12 encoder. Cohere/wikipedia-22-12-en-embeddings. Accessed: 2025-01-01
work page 2024
- [3]
-
[4]
MeMemo: RAG and Vector Search in Your Browser
2025. MeMemo: RAG and Vector Search in Your Browser. https://poloclub.github. io/mememo/. Accessed: 2025-01-01
work page 2025
-
[5]
SemanticFinder: frontend-only live semantic search with transformers.js
2025. SemanticFinder: frontend-only live semantic search with transformers.js. https://github.com/do-me/SemanticFinder. Accessed: 2025-01-01
work page 2025
-
[6]
Voy: A WASM vector similarity search written in Rust
2025. Voy: A WASM vector similarity search written in Rust. https://github.com/ tantaraio/voy. Accessed: 2025-01-01
work page 2025
-
[7]
Why IndexedDB is slow and what to use instead
2025. Why IndexedDB is slow and what to use instead. https://rxdb.info/slow- indexeddb.html. Accessed: 2025-01-01
work page 2025
- [8]
Show all 45 references
-
[9]
Erik Bernhardsson. 2017. Approximate Nearest Neighbors in C++/Python opti- mized for memory usage and loading/saving to disk. https://github.com/spotify/ annoy. Accessed: 2025-01-01
2017
-
[10]
Harrison Chase. 2022. LangChain. https://github.com/langchain-ai/langchain
2022
-
[11]
Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. 2021. Spann: Highly-efficient billion-scale approximate nearest neighborhood search. Advances in Neural Information Processing Systems (2021)
2021
-
[12]
Neo Christopher Chung, George Dyer, and Lennart Brocki. 2023. Chal- lenges of large language models for mental health counseling. arXiv preprint arXiv:2311.13857 (2023)
2023 arXiv
-
[13]
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The faiss library. arXiv preprint arXiv:2401.08281 (2024)
2024 arXiv
-
[14]
Fiona Draxler, Daniel Buschek, Mikke Tavast, Perttu Hämäläinen, Albrecht Schmidt, Juhi Kulshrestha, and Robin Welsch. 2023. Gender, age, and technology education influence the adoption and appropriation of LLMs. arXiv preprint arXiv:2310.06556 (2023)
2023 arXiv
-
[15]
Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013. Optimized product quantization. IEEE transactions on pattern analysis and machine intelligence (TPAMI) (2013), 744–755
2013
-
[16]
Samira Ghodratnama and Mehrdad Zakershahrak. 2023. Adapting LLMs for Efficient, Personalized Information Retrieval: Methods and Implications. In Pro- ceedings of the International Conference on Service-Oriented Computing (ICSOC 2023). 17–26
2023
-
[17]
W3C Community Group. 2025. WebAssembly. https://webassembly.org/. Ac- cessed: 2025-01-01
2025
-
[18]
Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020. Embedding- based retrieval in facebook search. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery &...
2020
-
[19]
Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence (TPAMI) (2010), 117–128
2010
-
[20]
Chao Jin, Zili Zhang, Xuanlin Jiang, Fangyue Liu, Xin Liu, Xuanzhe Liu, and Xin Jin. 2024. RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation. arXiv preprint arXiv:2404.12457 (2024)
2024 arXiv
-
[21]
Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data (2019), 535–547
2019
-
[22]
Andrew Kane. 2021. Pgvector: Open-source Vector Similarity Search for Postgres. https://github.com/pgvector/pgvector
2021
-
[23]
Haitao Li, Qingyao Ai, Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Zheng Liu, and Zhao Cao. 2023. Constructing tree-based index for efficient and effective dense retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrie...
2023
-
[24]
Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: A Python Toolkit for Reproducible Infor- mation Retrieval Research with Sparse and Dense Representations. InProceedings of the 44th International ACM SIGIR Conference...
2021
-
[25]
Malkov and D
Yu A. Malkov and D. A. Yashunin. 2020. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2020), 824–836
2020
-
[26]
mdn web docs. 2024. Service Worker API. https://developer.mozilla.org/en- US/docs/Web/API/Service_Worker_API. Accessed: 2025-01-01
2024
-
[27]
mdn web docs. 2025. Storage quotas and eviction criteria. https: //developer.mozilla.org/en-US/docs/Web/API/Storage_API/Storage_quotas_ and_eviction_criteria. Accessed: 2025-01-01
2025
-
[28]
mdn web docs. 2025. Wasm currently only allows 32-bit addressing. https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/ Memory/Memory. Accessed: 2025-01-01
2025
-
[29]
OpenAI. 2025. ChatGPT. https://chatgpt.com/. Accessed: 2025-01-01
2025
-
[30]
Ninh Pham and Tao Liu. 2022. Falconn++: A locality-sensitive filtering approach for approximate nearest neighbor search. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS 2022) (2022), 31186–31198
2022
-
[31]
LinQ AI Research. 2024. FinDER: Financial Document Retrieval Dataset. https: //huggingface.co/datasets/Linq-AI-Research/FinanceRAG. Accessed: 2025-01-01
2024
-
[32]
Korakit Seemakhupt, Sihang Liu, and Samira Khan. 2024. EdgeRAG: Online- Indexed RAG for Edge Devices. arXiv preprint arXiv:2412.21023 (2024)
2024 arXiv
-
[33]
Sina Semnani, Violet Yao, Heidi Zhang, and Monica Lam. 2023. WikiChat: Stop- ping the Hallucination of Large Language Model Chatbots by Few-Shot Ground- ing on Wikipedia. In Findings of the Association for Computational Linguistics: EMNLP 2023. 2387–2413
2023
-
[34]
Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kalu- arachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering. Transactions of the Association for Co...
2023
-
[35]
Yukihiro Tagami. 2017. Annexml: Approximate nearest neighbor search for extreme multi-label classification. In Proceedings of the 23rd ACM SIGKDD inter- national conference on knowledge discovery and data mining (KDD 2017) . 455–464
2017
-
[36]
Bing Tian, Haikun Liu, Yuhang Tang, Shihai Xiao, Zhuohui Duan, Xiaofei Liao, Xuecang Zhang, Junhua Zhu, and Yu Zhang. 2024. FusionANNS: An Efficient CPU/GPU Cooperative Processing Architecture for Billion-scale Approximate Nearest Neighbor Search. arXiv preprint arXiv:2409.165...
2024 arXiv
-
[37]
Wang and Duen Horng Chau
Zijie J. Wang and Duen Horng Chau. 2024. MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2024) . 2765–2770
2024
-
[38]
Lukas Wutschitz, Boris Köpf, Andrew Paverd, Saravan Rajmohan, Ahmed Salem, Shruti Tople, Santiago Zanella-Béguelin, Menglin Xia, and Victor Rühle. 2023. Rethinking privacy in machine learning pipelines from an information flow control perspective. arXiv preprint arXiv:2311.157...
2023 arXiv
-
[39]
JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang
-
[40]
Jianjin Zhang, Zheng Liu, Weihao Han, Shitao Xiao, Ruicheng Zheng, Yingxia Shao, Hao Sun, Hanqing Zhu, Premkumar Srinivasan, Weiwei Deng, et al. 2022. Uni-retriever: Towards learning the unified embedding based retriever in bing sponsored search. In Proceedings of the 28th ACM...
2022
-
[41]
Yanhao Zhang, Pan Pan, Yun Zheng, Kang Zhao, Yingya Zhang, Xiaofeng Ren, and Rong Jin. 2018. Visual search at alibaba. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (KDD 2018). 993–1001
2018
-
[42]
Zili Zhang, Chao Jin, Linpeng Tang, Xuanzhe Liu, and Xin Jin. 2023. Fast, Ap- proximate Vector Queries on Very Large Unstructured Datasets. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23) . 995– 1011
2023
-
[43]
Zili Zhang, Fangyue Liu, Gang Huang, Xuanzhe Liu, and Xin Jin. 2024. Fast Vector Query Processing for Large Datasets Beyond GPU Memory with Re- ordered Pipelining. In 21st USENIX Symposium on Networked Systems Design and Implementation, NSDI 2024, Santa Clara, CA, April 15-17,...
2024
-
[44]
Shuyan Zhou, Uri Alon, Frank F Xu, Zhengbao Jiang, and Graham Neubig. 2023. DocPrompting: Generating Code by Retrieving the Docs. In Proceedings of the Eleventh International Conference on Learning Representations (ICLR 2023)
2023
-
[2023]
In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI 2023)
Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI 2023) . 1–21
2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.