Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

OrchANN claims that unifying routing, indexing, and pruning at every stage of out-of-core vector search cuts SSD reads and delivers up to 17.2x higher throughput and 25.0x lower latency at matched recall.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

OrchANN's unified I/O orchestration—hybrid per-cluster indexes, query-driven routing graphs, and triangle-inequality pruning—cuts SSD reads and outperforms DiskANN, Starling, SPANN, and PipeANN in out-of-core vector search.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection Real design, flawed headline: the memory cap isn't applied consistently enough to support the 17–25x claims yet. the 3 major comments →

arxiv 2512.22838 v2 pith:36QBZFUL submitted 2025-12-28 cs.DB cs.IR

OrchANN: Hierarchical Orchestration for Skewed Out-of-Core Vector Search

Lizheng Chen , Pinhuan Wang , Shaonan Ma , Jie Zhang , Qing Wang , Zhengyi Yang , Heng Zhang , Mingxing Zhang
show 4 more authors
Fan Yang Guihai Chen Chen Tian Chengying Huan
This is my paper
classification cs.DB cs.IR
keywords approximate nearest neighbor searchout-of-core indexingSSD I/Osemantic skewnesshybrid local indexquery-aware routingtriangle inequality pruningvector search systems
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the performance of out-of-core approximate nearest neighbor search is governed less by raw hardware speed than by how many SSD reads the system decides to make. It claims that skewed semantic embeddings—where clusters vary wildly in size—break the standard recipe of uniform indexes, static routing, and product-quantization filtering, causing avoidable fetch-to-discard I/O. OrchANN therefore coordinates three tiers: per-cluster index selection by an offline profiler, a query-aware in-memory navigation graph that refreshes hot regions, and multi-level pruning with triangle-inequality bounds that rejects candidates before their raw vectors are read from disk. If correct, the result is an out-of-core engine that preserves recall while using far fewer SSD accesses, translating to up to 17.2x higher QPS and 25.0x lower latency on skewed workloads and substantial gains even on conventional datasets. A reader should care because retrieval-augmented LLM services and billion-scale vector databases operate precisely in this out-of-core, skewed regime.

Core claim

OrchANN's central claim is that the route-access-verify pipeline of an out-of-core query should be governed as one coordinated I/O-avoidance system. It assigns each skewed cluster a local index type chosen by an offline profiler under a global memory budget; it routes queries through an in-memory graph abstraction refreshed with query-trajectory hot regions; and it prunes before fetching, using triangle-inequality bounds with free pivots (edge distances for graphs, centroid distances for scans) to reject candidates that cannot enter the top-k. The paper reports that this cuts disk accesses by 3.5–7x at matched recall, keeps I/O growth under 10% as recall climbs, and beats four out-of-core ba

What carries the argument

The load-bearing mechanism is the I/O orchestration model: split query latency into route, access, and verify, and apply a different read-avoidance technique at each. The decisive object is the pre-fetch rejection test—a triangle-inequality lower bound with pivots that cost nothing extra (edge distances for graphs, centroid distances for scans). If the bound exceeds the current kth distance, the raw vector is never fetched. Hybrid index selection (an auto-profiler solving a memory-budgeted allocation) and the query-aware dynamic graph abstraction (a routing graph refreshed by search-depth hot regions) support it, but the triangle-inequality bound is what makes filtering exact rather than los

Load-bearing premise

The central comparison assumes all systems were actually squeezed into the same strict 4 GB memory budget; if any baseline ran with far more memory (one reported an 82.9 GB peak) or was truncated at a different threshold, the headline speedups are not like-for-like.

What would settle it

Run a like-for-like memory audit: enforce the same 4 GB cgroup limit on every system, log peak RSS and page faults, and count actual SSD bytes read per query at matched recall. If a baseline under that cap achieves QPS within a small factor of OrchANN's, or if OrchANN's I/O savings disappear, the central claim fails. Also, if on a manually balanced (non-skewed) dataset the QPS gain over the best baseline drops to near 1x, the skew-specific narrative would be falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Billion-scale search becomes feasible on a single machine with a few gigabytes of RAM and no vector replication: each vector is stored once and the routing graph stays near 100 MB.
  • The fetch-to-discard pattern is avoidable: with strict lower bounds, disk I/O per query grows by less than 10% as recall@10 moves from 90% to 98%.
  • Skewed RAG-style corpora need no rebalancing or duplicate storage; fixing the partition layout and adapting per-cluster indexes recovers most of the benefit.
  • Retrieval is not the RAG bottleneck: end-to-end LLM pipelines show retrieval under 6 ms, so faster vector search lowers serving cost.
  • Query-aware routing is learnable online: a bounded, lock-free refresh of the navigation graph improves probing precision without global rebuilds.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The bound-before-fetch principle should transfer to other out-of-core search settings—sparse retrieval, graph traversal, hybrid filters—wherever a cheap pivot can yield a distance lower bound; a testable extension is to use any available metadata column as a pivot.
  • The auto-profiler is a portable recipe: only bandwidth/latency primitives need recalibration for different storage (SATA SSD, NVMe, CXL), so hybrid index selection could be dropped into other vector engines as a black box.
  • The depth-weighted hot-node signal suggests routing-graph maintenance should track convergence depth (how late a node appears in search) rather than raw access frequency; dynamic indexing systems outside vector search may benefit from the same signal.
  • A decisive fairness test—re-running all baselines under the same enforced memory cap with byte-level I/O accounting—would reveal how much of the speedup is orchestration and how much is baseline memory accounting; that experiment would sharpen the paper's applicability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents OrchANN, an out-of-core ANNS engine designed for skewed workloads. It proposes three integrated mechanisms: (1) auto-profiler-guided hybrid per-cluster indexing that selects among Flat, graph, and IVF structures under a global memory budget; (2) a query-aware dynamic in-memory graph abstraction for routing, updated with hot-region nodes via count-min sketches and atomic snapshot swaps; and (3) multi-level pruning that combines cluster reordering/early stopping and triangle-inequality-based vector rejection before SSD fetches. The system is evaluated on five datasets (SIFT, DEEP, SPACEV, HotpotQA, TriviaQA) against DiskANN, Starling, SPANN, and PipeANN, reporting up to 17.2x higher QPS and 25.0x lower latency than baselines, reduced I/O, lower storage overhead, and an end-to-end RAG integration.

Significance. If substantiated, OrchANN would be a useful contribution to out-of-core vector search: it identifies a real weakness in current systems under semantic skewness and proposes a coherent end-to-end I/O orchestration view. The paper's strengths include concrete motivation with skewness measurements, a sensible three-tier cost model, ablations isolating each mechanism (Fig. 18), and evaluation on both standard and semantic workloads. However, the headline comparison is currently undermined by inconsistent memory-constraint accounting and mixed I/O metrics, and the vector-level pruning mechanism has an unresolved logical point about graph expansion. The core ideas are plausible; with a corrected experimental protocol and a clarified pruning algorithm, this could be a solid systems contribution.

major comments (3)
  1. [Section 6 / §6.8 / Table 4 / §6.3] The evaluation does not currently support the 'strict memory constraints' claim. Section 6 (Settings) states that all experiments use a 4 GB cgroup memory constraint and that peak RSS is reported. Section 6.8 and Table 4 then report SPANN peak memory of 82,916 MB and state 'any usage beyond 6 GB is subject to truncation'. These are mutually inconsistent: a 4 GB cgroup cannot admit an 82.9 GB RSS, and the 6 GB truncation threshold does not match 4 GB. Section 6.3 adds a third regime, with OrchANN at a 5 GB cap and DiskANN at 10 GB on SPACEV. Because the headline 17.2x/25.0x improvement is specifically against SPANN, it is not established that all systems ran under the same strict memory envelope. Please rerun SPANN under a true 4 GB cgroup and report peak RSS under that limit, or explicitly report per-system memory budgets and qualify the 'strict memory constraints' wording throughout.
  2. [Section 6.2] The I/O comparison mixes metrics: 'For DiskANN and Starling, we count system-level page reads; for OrchANN and SPANN ... we report page faults as a proxy for SSD accesses.' Page reads and page faults are not interchangeable; page faults depend on mapping, prefetching, and memory pressure, and are not a direct device-level read count. The claim of 3.5x fewer accesses than Starling and 7x fewer than DiskANN is therefore not supported as stated. Please report a uniform metric (e.g., bytes read or 4 KB reads at the block layer) for all systems, and state how SPANN's memory truncation affected its page-fault count.
  3. [Section 5.3 / Algorithm 1] The triangle-inequality pruning description states that a pruned neighbor v_j 'can still be enqueued ... so that search can continue exploring beyond it if needed', while its raw vector is never fetched. To expand v_j as a pivot for subsequent triangle-inequality bounds, one needs the exact D(q,v_j), which requires fetching v_j's raw vector. If pruned nodes are not expanded, recall may suffer because high-degree hubs can be pruned; if they are expanded, the 'decision to avoid loading its raw vector is final' is false. Please clarify how expansion of pruned nodes is implemented (e.g., separate metadata loads or deferred fetches) and quantify recall and I/O behavior for this path.
minor comments (6)
  1. [Section 3.4 / Fig. 18] Typographical issues: 'guaranties' in Observation 3 should be 'guarantees'; Fig. 18 labels 'NoCluterPrune' and 'NoVectorPrune' should be 'NoClusterPrune' and 'NoVectorPrune'.
  2. [References] References [3] and [4] are duplicate entries for SPTAG; [13] and [14] both cite the same NSG preprint. Please deduplicate.
  3. [Section 6.3] Please explicitly state why DiskANN was configured with a 10 GB budget while OrchANN used 5 GB on SPACEV. Even if this is intentional, it should be stated clearly so readers can interpret the comparison.
  4. [Section 5.1] The latency model introduces constants a, b, alpha_flat, beta_scan, and rho_cache but does not report how they were calibrated or validated against measured per-cluster latencies. A brief calibration table or sensitivity study would strengthen reproducibility.
  5. [General] No code or data availability statement is included. For a systems paper, releasing the implementation would materially aid reproducibility.
  6. [Table 2 / text] The dataset name is given as 'SpaceV1B' in Table 2 and 'SPACEV' elsewhere; please unify. Also, the storage breakdown categories in Fig. 17 (Head Index, FullLists, DiskIndex, ClusterData, LocalIndex) are not defined in the text.

Circularity Check

0 steps flagged

No significant circularity: OrchANN's claims are empirical measurements and system-engineering results, not derivations that reduce to their inputs.

full rationale

This is a systems paper whose central claims are benchmark measurements (QPS, latency, SSD I/O) under stated memory constraints, not algebraically derived predictions. The auto-profiler cost model in Section 5.1 contains calibrated constants (alpha, beta, a, b, rho) and solves a per-cluster index-selection allocation problem, but the reported performance numbers are independent empirical measurements against external baselines; no headline QPS/latency figure is mathematically forced by a fitted parameter. The triangle-inequality pruning in Section 5.3 relies on a standard mathematical fact, and although reference [2] has an overlapping author, the bound is independently verifiable and the paper's ablation (Section 6.5) empirically measures pruning's contribution rather than importing it by citation. The query-aware dynamic graph and hybrid indexing are evaluated through ablations and comparisons, so they are not defined in terms of the outcomes they are claimed to produce. The reader's memory-constraint concern (Section 6 sets a 4 GB cgroup, while Table 4 reports SPANN peak memory of 82,916 MB and mentions a 6 GB truncation threshold) is an experimental-fairness / correctness risk, not a circularity of derivation. No self-definitional step, fitted-input-called-prediction step, self-citation load-bearing step, or renaming of a known result as a new prediction was found. Honest non-finding is appropriate: the paper is self-contained as an empirical systems evaluation.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No new physical or mathematical entities are introduced. The paper's central claims rest on calibration parameters and domain assumptions about workload locality and metric-space properties. The most fragile assumptions are the proxy for SSD I/O and the consistency of memory limits across baselines.

free parameters (6)
  • α_flat (Flat scan compute coefficient) = not reported
    Introduced in T_flat(N) ≈ Tr(4Nd) + α_flat·N·C_vec (Section 5.1); captures SIMD/cache implementation effects, tuned offline.
  • a, b (hop-count model coefficients) = not reported
    H(N)=max{1, a·log N + b} estimates graph traversal hops for the latency model; fit to device/cluster behavior.
  • β_scan (IVF scan coefficient) = not reported
    In T_ivf(N), models non-ideal layout and prefetching effects; calibrated offline.
  • ρ_cache (graph cached-node ratio) = not reported
    M_graph(N)≈ρ_cache·N·B_node; chosen for the memory model in Section 5.1.
  • early-stop ratio ρ = not reported
    n=⌈ρM⌉ controls cluster-level early termination (Section 5.3); chosen by hand.
  • update period ΔQ and hot-set size h = not reported
    Control when and how many nodes are refreshed in the graph abstraction (Algorithm 1); tuned for stability.
axioms (5)
  • standard math Euclidean distances satisfy the triangle inequality, enabling lower-bound pruning.
    Used in Section 5.3 for vector pruning before fetch.
  • domain assumption IVF clusters and their size/occupancy are meaningful proxies for search cost and query hit probability under skewness.
    The motivation in Section 3 relies on cluster-size skewness translating directly into I/O inefficiency.
  • domain assumption Queries exhibit temporal locality / hot regions that can be captured by online frequency and convergence scores.
    The dynamic graph abstraction (Section 5.2) depends on stable hot-region distributions over time.
  • domain assumption Page faults from memory-mapped lazy loading are a valid proxy for SSD accesses when compared with system-level page reads.
    I/O comparison in Section 6.2 mixes these metrics across systems.
  • domain assumption The hardware cost model (bandwidth, latency, compute) predicts relative performance of Flat/graph/IVF indices across cluster sizes.
    The auto-profiler in Section 5.1 selects per-cluster index types based on this model.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of OrchANN: Hierarchical Orchestration for Skewed Out-of-Core Vector Search." pith.science (2026). https://pith.science/paper/36QBZFUL

@misc{pith2026251222838,
  author       = {Pith},
  title        = {Pith review of: OrchANN: Hierarchical Orchestration for Skewed Out-of-Core Vector Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/36QBZFUL}},
  note         = {Machine review of arXiv:2512.22838}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

At billion scale, approximate nearest neighbor search (ANNS) often becomes an out-of-core problem: the full vector collection and index structures exceed memory capacity, making query performance dominated by SSD accesses and DRAM-SSD data movement. Existing systems struggle to strike a balance between accuracy and efficiency: physical-overlap methods replicate vectors or index entries across partitions, enlarging the SSD-resident index and incurring extra I/O; quantization-based methods reduce memory usage, but their approximate distances are less reliable and often require costly raw-vector reranking from SSD to preserve recall. We present OrchANN (Orchestrated ANN Search), an out-of-core ANNS engine that orchestrates query routing, partition access, and query execution under tight memory constraints. OrchANN stores each cluster as a disjoint SSD partition with scale-aware indexes, while a memory-resident graph abstraction provides logical overlap before SSD access. During serving, OrchANN uses query hotness and cluster priorities from the graph abstraction to prune low-value clusters and improve access locality. Across five datasets under strict memory constraints, OrchANN delivers up to 17.2x higher QPS and 25.0x lower latency than state-of-the-art baselines, while preserving accuracy.

Figures

Figures reproduced from arXiv: 2512.22838 by Chengying Huan, Chen Tian, Fan Yang, Guihai Chen, Heng Zhang, Jie Zhang, Lizheng Chen, Mingxing Zhang, Pinhuan Wang, Qing Wang, Shaonan Ma, Zhengyi Yang.

Figure 1
Figure 1. Figure 1: Skewness on SIFT, HotpotQA, and TriviaQA. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: QPS across different indexes at Recall@10 > 95%. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Distribution. 0.60 0.65 0.70 0.75 0.80 0.85 Recall 0 1 2 3 4 QPS ×10 4 QPS vs Recall Strategy Centroid Graph Random Sample Ours [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: GT cluster percentage on SIFT and TriviaQA. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Rerank I/O. common navigation signals such as centroid routing [43] and ran￾dom sampling [47] are often too weak to make this decision accu￾rately [PITH_FULL_IMAGE:figures/full_fig_p004_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Workflow of OrchANN, including offline hybrid index construction and online query processing. [PITH_FULL_IMAGE:figures/full_fig_p005_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Graph abstraction guided cluster reordering. [PITH_FULL_IMAGE:figures/full_fig_p007_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Triangle inequality pruning within a cluster. [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: QPS overall evaluation on SIFT, DEEP, TriviaQA and HotpotQA datasets. [PITH_FULL_IMAGE:figures/full_fig_p009_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Latency overall evaluation on SIFT, DEEP, TriviaQA and HotpotQA datasets. [PITH_FULL_IMAGE:figures/full_fig_p009_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: QPS evaluation comparing with PipeANN. 82.5 85.0 87.5 90.0 92.5 95.0 97.5 Recall@10 (%) 200 400 600 800 Mean IOs Mean IOs vs Recall@10 OrchANN DiskANN Starling (a) SIFT: IO-Recall@10. 80.0 82.5 85.0 87.5 90.0 92.5 95.0 97.5 Recall@10 (%) 100 200 300 400 Mean IOs Mean IOs vs Recall@10 OrchANN DiskANN Starling (b) DEEP: IO-Recall@10 [PITH_FULL_IMAGE:figures/full_fig_p009_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Disk I/O comparison with various Recall@10. [PITH_FULL_IMAGE:figures/full_fig_p009_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Performance evaluation on billion-scale data. [PITH_FULL_IMAGE:figures/full_fig_p010_15.png] view at source ↗
Figure 18
Figure 18. Figure 18: Efficiency of hybrid indexing, graph abstraction, [PITH_FULL_IMAGE:figures/full_fig_p010_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Low-Latency Out-of-Core ANN Search in High-Dimensional Space

    cs.DB 2026-05 unverdicted novelty 5.0

    SkipDisk is a disk-memory hybrid ANN search that achieves 63-85% of HNSW latency at 10-20% memory footprint via dedicated pivots for tighter lower bounds, three-level pruning, and decoupled async I/O.

Reference graph

Works this paper leans on

60 extracted references · 7 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Microsoft copilot and anthropic claude ai in education and library service

    Adebowale Jeremy Adetayo, Mariam Oyinda Aborisade, and Basheer Abiodun Sanni. Microsoft copilot and anthropic claude ai in education and library service. Library Hi Tech News, 2024

    2024

  2. [2]

    Achieving sub-second pairwise query over evolving graphs

    Hongtao Chen, Mingxing Zhang, Ke Yang, Kang Chen, Albert Zomaya, Yongwei Wu, and Xuehai Qian. Achieving sub-second pairwise query over evolving graphs. InProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, pages 1–15, 2023

    2023

  3. [3]

    Qi Chen, Haidong Wang, Mingqin Li, Gang Ren, Scarlett Li, Jeffery Zhu, Jason Li, Chuanjie Liu, Lintao Zhang, and Jingdong Wang.SPTAG: A library for fast approximate nearest neighbor search, 2018

    2018

  4. [4]

    Sptag: A library for fast approximate nearest neighbor search, 2018

    Qi Chen, Haidong Wang, Mingqin Li, Gang Ren, Scarlett Li, Jeffery Zhu, Jason Li, Chuanjie Liu, Lintao Zhang, and Jingdong Wang. Sptag: A library for fast approximate nearest neighbor search, 2018

    2018

  5. [5]

    Spann: Highly-efficient billion-scale approximate nearest neighborhood search.Advances in Neural Information Processing Systems, 34:5199–5212, 2021

    Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. Spann: Highly-efficient billion-scale approximate nearest neighborhood search.Advances in Neural Information Processing Systems, 34:5199–5212, 2021

    2021

  6. [6]

    Memanns: Enhancing billion-scale anns efficiency with practical pim hardware.arXiv preprint arXiv:2410.23805, 2024

    Sitian Chen, Amelie Chi Zhou, Yucheng Shi, Yusen Li, and Xin Yao. Memanns: Enhancing billion-scale anns efficiency with practical pim hardware.arXiv preprint arXiv:2410.23805, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Onesparse: A unified system for multi-index vector search

    Yaoqi Chen, Ruicheng Zheng, Qi Chen, Shuotao Xu, Qianxi Zhang, Xue Wu, Weihao Han, Hua Yuan, Mingqin Li, Yujing Wang, et al. Onesparse: A unified system for multi-index vector search. InCompanion Proceedings of the ACM Web Conference 2024, pages 393–402, 2024

    2024

  8. [8]

    Magicpig: Lsh sampling for efficient llm generation.arXiv preprint arXiv:2410.16179, 2024

    Zhuoming Chen, Ranajoy Sadhukhan, Zihao Ye, Yang Zhou, Jianyu Zhang, Niklas Nolte, Yuandong Tian, Matthijs Douze, Leon Bottou, Zhihao Jia, et al. Magicpig: Lsh sampling for efficient llm generation.arXiv preprint arXiv:2410.16179, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Deep neural networks for youtube recommendations

    Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommendations. InProceedings of the 10th ACM conference on recommender systems, pages 191–198, 2016

    2016

  10. [10]

    The power of noise: Redefining retrieval for rag systems

    Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. The power of noise: Redefining retrieval for rag systems. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 719–729, 2024

    2024

  11. [11]

    Alayadb: The data foundation for efficient and effective long-context llm inference.arXiv preprint arXiv:2504.10326, 2025

    Yangshen Deng, Zhengxin You, Long Xiang, Qilong Li, Peiqi Yuan, Zhaoyang Hong, Yitao Zheng, Wanting Li, Runzhong Li, Haotian Liu, et al. Alayadb: The data foundation for efficient and effective long-context llm inference.arXiv preprint arXiv:2504.10326, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Risgraph: A real-time streaming system for evolving graphs to support sub-millisecond per-update analysis at millions ops/s

    Guanyu Feng, Zixuan Ma, Daixuan Li, Shengqi Chen, Xiaowei Zhu, Wentao Han, and Wenguang Chen. Risgraph: A real-time streaming system for evolving graphs to support sub-millisecond per-update analysis at millions ops/s. In Proceedings of the 2021 International Conference on Management of Data, pages 513–527, 2021

    2021

  13. [14]

    Fast approximate near- est neighbor search with the navigating spreading-out graph.arXiv preprint arXiv:1707.00143, 2017

    Cong Fu, Chao Xiang, Changxu Wang, and Deng Cai. Fast approximate near- est neighbor search with the navigating spreading-out graph.arXiv preprint arXiv:1707.00143, 2017

    Pith/arXiv arXiv 2017

  14. [15]

    Fast approximate nearest neighbor search with the navigating spreading-out graph.Proc

    Cong Fu, Chao Xiang, Changxu Wang, and Deng Cai. Fast approximate nearest neighbor search with the navigating spreading-out graph.Proc. VLDB Endow., 12(5):461–474, 2019

    2019

  15. [16]

    Practical and asymptotically optimal quantization of high- dimensional vectors in euclidean space for approximate nearest neighbor search

    Jianyang Gao, Yutong Gou, Yuexuan Xu, Yongyi Yang, Cheng Long, and Raymond Chi-Wing Wong. Practical and asymptotically optimal quantization of high- dimensional vectors in euclidean space for approximate nearest neighbor search. Proceedings of the ACM on Management of Data, 3(3):1–26, 2025

    2025

  16. [17]

    Rabitq: quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search.Proceedings of the ACM on Management of Data, 2(3):1–27, 2024

    Jianyang Gao and Cheng Long. Rabitq: quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search.Proceedings of the ACM on Management of Data, 2(3):1–27, 2024

    2024

  17. [18]

    Symphonyqg: Towards symphonious integration of quantization and graph for approximate nearest neighbor search.Proceedings of the ACM on Management of Data, 3(1):1–26, 2025

    Yutong Gou, Jianyang Gao, Yuexuan Xu, and Cheng Long. Symphonyqg: Towards symphonious integration of quantization and graph for approximate nearest neighbor search.Proceedings of the ACM on Management of Data, 3(1):1–26, 2025

    2025

  18. [19]

    Ggnn: Graph-based gpu nearest neighbor search.IEEE Transactions on Big Data, 9(1):267–279, 2022

    Fabian Groh, Lukas Ruppert, Patrick Wieschollek, and Hendrik PA Lensch. Ggnn: Graph-based gpu nearest neighbor search.IEEE Transactions on Big Data, 9(1):267–279, 2022

    2022

  19. [20]

    Achieving {Low-Latency}{ Graph-Based} vector search via aligning{Best-First} search algorithm with{SSD}

    Hao Guo and Youyou Lu. Achieving {Low-Latency}{ Graph-Based} vector search via aligning{Best-First} search algorithm with{SSD}. In19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25), pages 171–186, 2025

    2025

  20. [21]

    Hedrarag: Co-optimizing generation and retrieval for hetero- geneous rag workflows

    Zhengding Hu, Vibha Murthy, Zaifeng Pan, Wanlu Li, Xiaoyi Fang, Yufei Ding, and Yuke Wang. Hedrarag: Co-optimizing generation and retrieval for hetero- geneous rag workflows. InProceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles, pages 623–638, 2025

    2025

  21. [22]

    In2023 USENIX Annual Technical Conference (USENIX ATC 23), pages 585–600, 2023

    Junhyeok Jang, Hanjin Choi, Hanyeoreum Bae, Seungjun Lee, Miryeong Kwon, and Myoungsoo Jung.{CXL-ANNS}:{Software-Hardware} collaborative mem- ory disaggregation and computation for{Billion-Scale} approximate nearest neighbor search. In2023 USENIX Annual Technical Conference (USENIX ATC 23), pages 585–600, 2023

    2023

  22. [23]

    Diskann: Fast accurate billion-point nearest neighbor search on a single node.Advances in neural information processing Systems, 32, 2019

    Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnawamy, and Rohan Kadekodi. Diskann: Fast accurate billion-point nearest neighbor search on a single node.Advances in neural information processing Systems, 32, 2019

    2019

  23. [24]

    Product quantization for nearest neighbor search.IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117–128, 2011

    Hervé Jégou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search.IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117–128, 2011. The ANN_SIFT1M dataset was introduced in this paper. Dataset available at http://corpus-texmex.irisa.fr/

    2011

  24. [25]

    Billion-scale similarity search with gpus.IEEE Transactions on Big Data, 7(3):535–547, 2019

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus.IEEE Transactions on Big Data, 7(3):535–547, 2019

    2019

  25. [26]

    Weld, and Luke Zettlemoyer

    Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551, 2017

    Pith/arXiv arXiv 2017

  26. [27]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, An- drew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research.Transa...

    2019

  27. [28]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, and Karpukhin. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    2020

  28. [29]

    Ansmet: Approximate nearest neighbor search with near-memory processing and hybrid early termination

    Yiwei Li, Yuxin Jin, Boyu Tian, Huanchen Zhang, and Mingyu Gao. Ansmet: Approximate nearest neighbor search with near-memory processing and hybrid early termination. InProceedings of the 52nd Annual International Symposium on Computer Architecture, pages 1093–1107, 2025

    2025

  29. [30]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024. 12

    Pith/arXiv arXiv 2024

  30. [31]

    Retrievalattention: Accelerating long-context llm inference via vector retrieval.arXiv preprint arXiv:2409.10516, 2024

    Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Zhenhua Han, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, et al. Retrievalattention: Accelerating long-context llm inference via vector retrieval.arXiv preprint arXiv:2409.10516, 2024

    Pith/arXiv arXiv 2024

  31. [32]

    Malkov and D

    Yu A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neigh- bor search using hierarchical navigable small world graphs.IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):824–836, 2020

    2020

  32. [33]

    Parlayann: Scalable and deter- ministic parallel graph-based approximate nearest neighbor search algorithms

    Magdalen Dobson Manohar, Zheqi Shen, Guy Blelloch, Laxman Dhulipala, Yan Gu, Harsha Vardhan Simhadri, and Yihan Sun. Parlayann: Scalable and deter- ministic parallel graph-based approximate nearest neighbor search algorithms. InProceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, pages 270–285, 2024

    2024

  33. [34]

    Ilyas, Theodoros Rekatsinas, and Shivaram Venkataraman

    Jason Mohoney, Devesh Sarda, Mengze Tang, Shihabur Rahman Chowdhury, Anil Pacaci, Ihab F. Ilyas, Theodoros Rekatsinas, and Shivaram Venkataraman. Quake: Adaptive indexing for vector search. In Lidong Zhou and Yuanyuan Zhou, editors, 19th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2025, Boston, MA, USA, July 7-9, 2025, pages 153–...

    2025

  34. [35]

    Exploring dis- tributed vector databases performance on hpc platforms: A study with qdrant

    Seth Ockerman, Amal Gueroudji, Song Young Oh, Robert Underwood, Nicholas Chia, Kyle Chard, Robert Ross, and Shivaram Venkataraman. Exploring dis- tributed vector databases performance on hpc platforms: A study with qdrant. InProceedings of the SC’25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis,...

    2025

  35. [36]

    Embedding- based news recommendation for millions of users

    Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. Embedding- based news recommendation for millions of users. InProceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1933–1942, 2017

    1933

  36. [37]

    Cagra: Highly parallel graph construction and approximate nearest neighbor search for gpus

    Hiroyuki Ootomo, Akira Naruse, Corey Nolet, Ray Wang, Tamas Feher, and Yong Wang. Cagra: Highly parallel graph construction and approximate nearest neighbor search for gpus. In2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 4236–4247. IEEE, 2024

    2024

  37. [38]

    Openai developer platform, 2025

    OpenAI. Openai developer platform, 2025

    2025

  38. [39]

    Vector database management techniques and systems

    James Jie Pan, Jianguo Wang, and Guoliang Li. Vector database management techniques and systems. InCompanion of the 2024 International Conference on Management of Data, pages 597–604, 2024

    2024

  39. [40]

    Chatgpt and open-ai models: A preliminary review.Future Internet, 15(6):192, 2023

    Konstantinos I Roumeliotis and Nikolaos D Tselikas. Chatgpt and open-ai models: A preliminary review.Future Internet, 15(6):192, 2023

    2023

  40. [41]

    DiskANN: Graph-structured Indices for Scalable, Fast, Fresh and Filtered Approximate Nearest Neighbor Search, 2023

    Harsha Vardhan Simhadri, Ravishankar Krishnaswamy, Gopal Srinivasa, Suhas Jayaram Subramanya, Andrija Antonijevic, Dax Pryce, David Kaczyn- ski, Shane Williams, Siddarth Gollapudi, Varun Sivashankar, Neel Karia, Aditi Singh, Shikhar Jaiswal, Neelam Mahapatro, Philip Adams, Bryan Tower, and Yash Patel. DiskANN: Graph-structured Indices for Scalable, Fast, ...

    2023

  41. [42]

    Scalable billion-point approximate nearest neighbor search using{SmartSSDs}

    Bing Tian, Haikun Liu, Zhuohui Duan, Xiaofei Liao, Hai Jin, and Yu Zhang. Scalable billion-point approximate nearest neighbor search using{SmartSSDs}. In2024 USENIX Annual Technical Conference (USENIX ATC 24), pages 1135–1150, 2024

    2024

  42. [43]

    Towards high-throughput and low-latency billion-scale vector search via{CPU/GPU} collaborative filter- ing and re-ranking

    Bing Tian, Haikun Liu, Yuhang Tang, Shihai Xiao, Zhuohui Duan, Xiaofei Liao, Hai Jin, Xuecang Zhang, Junhua Zhu, and Yu Zhang. Towards high-throughput and low-latency billion-scale vector search via{CPU/GPU} collaborative filter- ing and re-ranking. In23rd USENIX Conference on File and Storage Technologies (FAST 25), pages 171–185, 2025

    2025

  43. [44]

    Mirage-anns: Mixed approach graph-based indexing for approximate nearest neighbor search.Proceedings of the ACM on Management of Data, 3(3):1–27, 2025

    Sairaj Voruganti and M Tamer Özsu. Mirage-anns: Mixed approach graph-based indexing for approximate nearest neighbor search.Proceedings of the ACM on Management of Data, 3(3):1–27, 2025

    2025

  44. [45]

    Milvus: A purpose-built vector data management system

    Jianguo Wang, Xiaomeng Yi, Rentong Guo, Hai Jin, Peng Xu, Shengjun Li, Xi- angyu Wang, Xiangzhou Guo, Chengming Li, Xiaohai Xu, et al. Milvus: A purpose-built vector data management system. InProceedings of the 2021 Inter- national Conference on Management of Data, pages 2614–2627, 2021

    2021

  45. [46]

    Accelerating graph indexing for anns on modern cpus.Proceedings of the ACM on Management of Data, 3(3):1–29, 2025

    Mengzhao Wang, Haotian Wu, Xiangyu Ke, Yunjun Gao, Yifan Zhu, and Wenchao Zhou. Accelerating graph indexing for anns on modern cpus.Proceedings of the ACM on Management of Data, 3(3):1–29, 2025

    2025

  46. [47]

    Mengzhao Wang, Weizhi Xu, Xiaomeng Yi, Songlin Wu, Zhangyang Peng, Xi- angyu Ke, Yunjun Gao, Xiaoliang Xu, Rentong Guo, and Charles Xie. Starling: An i/o-efficient disk-resident graph index framework for high-dimensional vector similarity search on data segment.Proceedings of the ACM on Management of Data, 2(1):1–27, 2024

    2024

  47. [48]

    Deltapq: lossless product quantization code compression for high dimensional similarity search.Proceedings of the VLDB Endowment, 13(13):3603–3616, 2020

    Runhui Wang and Dong Deng. Deltapq: lossless product quantization code compression for high dimensional similarity search.Proceedings of the VLDB Endowment, 13(13):3603–3616, 2020

    2020

  48. [49]

    Det-lsh: a locality- sensitive hashing scheme with dynamic encoding tree for approximate nearest neighbor search.arXiv preprint arXiv:2406.10938, 2024

    Jiuqi Wei, Botao Peng, Xiaodong Lee, and Themis Palpanas. Det-lsh: a locality- sensitive hashing scheme with dynamic encoding tree for approximate nearest neighbor search.arXiv preprint arXiv:2406.10938, 2024

    arXiv 2024

  49. [50]

    Turbocharge anns on real processing-in-memory by enabling fine-grained per-pim-core scheduling

    Puqing Wu, Minhui Xie, Enrui Zhao, Dafang Zhang, Jing Wang, Xiao Liang, Kai Ren, and Yunpeng Chai. Turbocharge anns on real processing-in-memory by enabling fine-grained per-pim-core scheduling

  50. [51]

    C-pack: Pack- aged resources to advance general chinese embedding, 2023

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Pack- aged resources to advance general chinese embedding, 2023

    2023

  51. [52]

    Tribase: A vector data query engine for reliable and lossless pruning compression using triangle inequalities.Proceedings of the ACM on Management of Data, 3(1):1–28, 2025

    Qian Xu, Juan Yang, Feng Zhang, Junda Pan, Kang Chen, Youren Shen, Amelie Chi Zhou, and Xiaoyong Du. Tribase: A vector data query engine for reliable and lossless pruning compression using triangle inequalities.Proceedings of the ACM on Management of Data, 3(1):1–28, 2025

    2025

  52. [53]

    Spfresh: Incremental in-place update for billion-scale vector search

    Yuming Xu, Hengyu Liang, Jin Li, Shuotao Xu, Qi Chen, Qianxi Zhang, Cheng Li, Ziyue Yang, Fan Yang, Yuqing Yang, et al. Spfresh: Incremental in-place update for billion-scale vector search. InProceedings of the 29th Symposium on Operating Systems Principles, pages 545–561, 2023

    2023

  53. [54]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. InConference on Empirical Methods in Natural Language Processing (EMNLP), 2018

    2018

  54. [55]

    Cacheblend: Fast large language model serving for rag with cached knowledge fusion

    Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, and Junchen Jiang. Cacheblend: Fast large language model serving for rag with cached knowledge fusion. InProceedings of the Twentieth European Conference on Computer Systems, pages 94–109, 2025

    2025

  55. [56]

    Gpu-accelerated proximity graph approximate nearest neighbor search and construction

    Yuanhang Yu, Dong Wen, Ying Zhang, Lu Qin, Wenjie Zhang, and Xuemin Lin. Gpu-accelerated proximity graph approximate nearest neighbor search and construction. In2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 552–564. IEEE, 2022

    2022

  56. [57]

    Pqcache: Product quantization-based kvcache for long context llm inference.Proceedings of the ACM on Management of Data, 3(3):1–30, 2025

    Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, and Bin Cui. Pqcache: Product quantization-based kvcache for long context llm inference.Proceedings of the ACM on Management of Data, 3(3):1–30, 2025

    2025

  57. [58]

    Uni- retriever: Towards learning the unified embedding based retriever in bing spon- sored search

    Jianjin Zhang, Zheng Liu, Weihao Han, Shitao Xiao, Ruicheng Zheng, Yingxia Shao, Hao Sun, Hanqing Zhu, Premkumar Srinivasan, Weiwei Deng, et al. Uni- retriever: Towards learning the unified embedding based retriever in bing spon- sored search. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4493–4501, 2022

    2022

  58. [59]

    {VBASE}: Unifying online vector similarity search and relational queries via relaxed monotonicity

    Qianxi Zhang, Shuotao Xu, Qi Chen, Guoxin Sui, Jiadong Xie, Zhizhen Cai, Yaoqi Chen, Yinxuan He, Yuqing Yang, Fan Yang, et al. {VBASE}: Unifying online vector similarity search and relational queries via relaxed monotonicity. In17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23), pages 377–395, 2023

    2023

  59. [60]

    Fast vector query processing for large datasets beyond {GPU} memory with reordered pipelining

    Zili Zhang, Fangyue Liu, Gang Huang, Xuanzhe Liu, and Xin Jin. Fast vector query processing for large datasets beyond {GPU} memory with reordered pipelining. In21st USENIX Symposium on Networked Systems Design and Imple- mentation (NSDI 24), pages 23–40, 2024

    2024

  60. [61]

    Gts: Gpu-based tree index for fast similarity search.Proceedings of the ACM on Management of Data, 2(3):1–27, 2024

    Yifan Zhu, Ruiyao Ma, Baihua Zheng, Xiangyu Ke, Lu Chen, and Yunjun Gao. Gts: Gpu-based tree index for fast similarity search.Proceedings of the ACM on Management of Data, 2(3):1–27, 2024. 13

    2024

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.