REVIEW 4 major objections 4 minor 24 references
DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers
T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper argues that a single distributed graph index can beat partition-and-route architectures for web-scale approximate nearest-neighbor search.
desk verdict A credible production ANN serving architecture with a genuine 50B-vector deployment, but the 6x efficiency headline is only as strong as the single clustered-partitioning baseline it is measured against. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the single logical DISKANN graph served from a distributed key-value store, adapted by three mechanisms: (1) compressed vectors are duplicated into every graph node that references them, collapsing the many lookups of a beam-search iteration into one batched remote read; (2) an in-memory head index is built over the graph's top layers, giving a network-hop-free seeding step; (3) a node-scoring service runs on each storage host, computing distances locally and returning compact scores and candidate IDs. Together these turn a graph built for one machine's SSD into a distributed structure whose cost per query grows with graph hops, not with the number of partitions.
What would settle it
Run the same 50-billion-vector slice with the advanced graph-partitioning schemes the paper names as future work in Section 4.4, at the same machine footprint and the same 15k-QPS resource bound, and compare recall@5 and recall@200 at that throughput; if such a partitioner matches DISTRIBUTEDANN's recall or reaches comparable throughput at equal recall, the claimed 6x advantage is falsified.
Extended reading notes
Core claim
On its own terms, the paper claims that state-of-the-art performance on very-large-scale ANN search can be achieved through a single large logical index stored in a distributed key-value store. The authors take a DISKANN graph—normally designed for one machine's local SSD—and adapt it for network access by duplicating each vector's compressed representation into all graph nodes that point to it, so one beam-search hop needs only one batched read. They add a sharded in-memory head index over the graph's top layers so search starts from good candidates without paying network hops for the first iterations. They also push node scoring into a service running on each key-value store host, which re
Load-bearing premise
The 6x claim rests on the clustered-partitioning baseline being representative; the paper has not compared against the strongest graph-partitioning schemes and says these may close the gap.
Editorial extensions
If this is right
- Sublinear scaling: as the corpus grows, query cost is tied to graph hops rather than the number of partitions, so the cost curve flattens at web scale.
- Flexible IO allocation: a unified graph lets a query spend its IO budget where the nearest neighbours actually live, yielding higher recall at the same IO per query.
- Throughput scaling by sharding: adding hosts to the key-value store adds IOPS and CPU without increasing partitions searched per query, so the system is bound by SSD space rather than IO.
- Graceful degradation: partial failures of node-scoring requests cause recall loss roughly proportional to the failure rate, so operators can safely timeout stragglers.
- Head-index bottleneck: the CPU-bound head index is the current limiter; adding head replicas raises QPS without adding machines because total SSD space remains the bounding resource.
Reading between the lines
- If the advanced graph-partitioning baselines named in Section 4.4 close most of the gap, the durable contribution shifts from '6x faster' to 'one graph avoids the operational complexity of load-balancing many partitions under shifting traffic.'
- The space amplification from duplicated compressed vectors is the main economic cost; packing several nearby full-dimensional vectors into one graph node, which the paper flags as future work, is a concrete experiment that could reduce that cost substantially.
- If the head-index CPU bound is relieved by a GPU or kernel-bypass networking, the latency disadvantage relative to partitioned systems could shrink further without changing the graph layout.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents DISTRIBUTEDANN, a distributed ANN serving system that stores a single large DISKANN graph in a distributed key-value store, with three main modifications: duplicating compressed vector representations into graph nodes, maintaining an in-memory 'head index' over the top layers of the graph, and performing near-data scoring at each key-value host. The system is reported to serve a 50-billion-vector slice of the Bing web index at over 100,000 QPS with 26ms median latency, claimed to be 6x more efficient than a clustered-partitioning and routing baseline. The paper also describes a graph-stitching construction method, reliability experiments under partial node-scoring failures, and a deployment experience in Microsoft Bing.
Significance. If validated, the result is significant: it demonstrates that a single logical ANN graph can be scaled across thousands of machines in a shared-nothing cluster, avoiding the linear cost growth of partitioned-and-routed serving while reusing commodity key-value store infrastructure. The paper's principal strengths are its production scale, concrete measurements on a 50B-vector workload, the clear articulation of bandwidth and computation tradeoffs (Eq. 1-2), and an explicit discussion of failure modes. However, the central efficiency claim rests on a single baseline comparison and several experimental details are underspecified, so the contribution is currently conditional rather than fully established.
major comments (4)
- [§4, Table 1] The headline '6x more efficient' (Abstract and §4) rests on a single comparison in Table 1. DISTRIBUTEDANN is evaluated with H=5, BW=128, R=72, 640 IO/query, recall@200=71.9, and >100k QPS; the clustered-partitioning baseline searches 40 of 203 partitions with I=120 per partition, 4800 IO/query, recall@200=67.4, and ~15k QPS. The throughput ratio therefore conflates the serving architecture, the IO budget, and the achieved recall. The statement that parameters are chosen for 'similar footprint (by bounding resource) at 15k QPS' is not accompanied by a quantitative method for bounding. To support the 6x claim, the comparison should be at matched recall/IO, or the baseline should be demonstrated to be a strong representative of the state of the art.
- [§4.4] The paper concedes that advanced partitioning schemes (Dong et al. 2019; Gottesbüren et al. 2024) reduce cross-partition edges and 'may be preferable in very latency-constrained scenarios', and states that 'further work is needed' to compare them empirically. This directly limits the central claim that DISTRIBUTEDANN is '6x more efficient than existing partitioning and routing strategies'. Either include a stronger partitioning baseline, or explicitly restrict the claim to the specific clustered partitioner implemented here and to throughput-oriented serving.
- [§4, recall evaluation] Recall ground truth is not specified. For the 50B-vector slice, there is no description of how true nearest neighbors were computed (e.g., exhaustive search, ground-truth set size, sampling procedure). Without this, the recall values in Table 1 and Figure 4 cannot be reproduced or interpreted. Additionally, all throughput and latency numbers appear to be single-point measurements taken in a production environment with mixed workloads; the paper should report variance or repeated trials to support the reported QPS and latency figures.
- [§5, Conclusion] The conclusion states that DISTRIBUTEDANN 'achieves sublinear scaling of ANN search on very large datasets', but no experiment varies the dataset size or the number of machines. The evidence is one 50B-vector slice at a single configuration. While the log-index complexity argument is analytically plausible, the empirical claim of sublinear scaling across scales is not demonstrated. Please add a scaling study or soften the conclusion to a single-scale demonstration.
minor comments (4)
- [Abstract / §1] The abstract says '50 billion vector graph index' while §1 says 'fifty-billion vector subset of a web search dataset'. Clarify whether the evaluated index is a slice of a larger corpus and whether the multiplicity of partition replicas is included in the 50B count.
- [§2.2, Eq. (1)-(2)] The formulas for space amplification and bandwidth savings would benefit from clearer parentheses and explicit definitions of d, d_OPQ, and R. As written, the numerator and denominator are ambiguous.
- [§4.1] The sentence 'at 3 replicas becomes CPU-bound' is unclear: does this refer to the head index replicas or the node-scoring service replicas? Specify which replica count is being varied and what '3 replicas' corresponds to in Table 1.
- [Figure 4] The x-axis is labeled 'IO per Query (log scale)' but the figure caption does not state whether the plotted IO counts include the head-index search or only the graph-node reads. Please clarify the IO accounting across both systems.
Circularity Check
No significant circularity: the paper's headline numbers are empirical measurements, and the self-citations used are to externally validated building blocks, not to definitions or fitted targets.
full rationale
The paper's central claims—26ms median latency, >100k QPS, 6x efficiency, and recall improvements—are reported experimental results from a deployed system, not predictions derived from equations containing those targets. The only equations, Eqs. (1) and (2), compute space amplification and bandwidth savings from the chosen layout parameters; these are algebraic consequences of the design, not circular validations of the headline numbers. The self-citations to DISKANN and SPANN (Subramanya et al., 2019; Wang, 2021) are used as building blocks: DISKANN provides the single-node graph index and SPANN the graph-stitching technique. These are prior, externally published systems with their own empirical support, and the paper does not invoke a uniqueness theorem or a fitted parameter to force its conclusions. Section 4.4 explicitly acknowledges that advanced partitioning schemes may narrow or close the efficiency gap and calls for further comparison; this is a limitation regarding the strength of the baseline, not a circularity. The '6x' claim depends on the representativeness of the clustered-partitioning baseline, but that is a benchmarking validity concern, not a self-referential derivation. Overall, no step in the paper reduces by construction to its own inputs.
Assumptions & free parameters
free parameters (4)
- Graph degree R (DISTRIBUTEDANN) =
72 (truncated from 106)
- Head index size =
2.5 billion vectors
- Beam search parameters (BW, H) =
128, 5
- Baseline partition routing (top-N partitions) =
40 of 203
assumptions (4)
- domain assumption A stitched graph built from clustered partitions preserves enough graph quality to support beam search recall comparable to an incrementally built graph.
- domain assumption Random sharding of the KV store yields predictable traffic and allows throughput scaling by adding hosts without changing routing.
- domain assumption Recall on sampled web queries is measured against a valid ground truth, the construction of which is not described.
- domain assumption A head index built by BFS over the top layers of the graph provides entry points that reach the rest of the graph within H hops.
invented entities (2)
-
Head index
-
Near-data node scoring service
Cite this review
Pith. "Pith review of DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers." pith.science (2026). https://pith.science/paper/6727ZJZH
@misc{pith2026250906046,
author = {Pith},
title = {Pith review of: DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers},
year = {2026},
howpublished = {\url{https://pith.science/paper/6727ZJZH}},
note = {Machine review of arXiv:2509.06046}
}
read the original abstract
We present DISTRIBUTEDANN, a distributed vector search service that makes it possible to search over a single 50 billion vector graph index spread across over a thousand machines that offers 26ms median query latency and processes over 100,000 queries per second. This is 6x more efficient than existing partitioning and routing strategies that route the vector query to a subset of partitions in a scale out vector search system. DISTRIBUTEDANN is built using two well-understood components: a distributed key-value store and an in-memory ANN index. DISTRIBUTEDANN has replaced conventional scale-out architectures for serving the Bing search engine, and we share our experience from making this transition.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Andoni, A. and Indyk, P. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Communications of the ACM, 51 0 (1): 0 117--122, 2008
work page 2008
-
[3]
Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms, 2018
Aumüller, M., Bernhardsson, E., and Faithfull, A. Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms, 2018. URL https://arxiv.org/abs/1807.05614
arXiv 2018
-
[4]
Babenko, A. and Lempitsky, V. The inverted multi-index. IEEE transactions on pattern analysis and machine intelligence, 37 0 (6): 0 1247--1260, 2014
work page 2014
-
[5]
Big-ann benchmarks: Neurips 2023
Big-ANN Benchmarks . Big-ann benchmarks: Neurips 2023. https://big-ann-benchmarks.com/neurips23.html, 2023. Accessed: 2025-01-04
work page 2023
-
[6]
Dean, J. and Barroso, L. A. The tail at scale. Communications of the ACM, 56 0 (2): 0 74--80, 2013
work page 2013
-
[7]
Dynamo: Amazon's highly available key-value store
DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., and Vogels, W. Dynamo: Amazon's highly available key-value store. ACM SIGOPS operating systems review, 41 0 (6): 0 205--220, 2007
work page 2007
-
[8]
Dong, Y., Indyk, P., Razenshteyn, I. P., and Wagner, T. Learning space partitions for nearest neighbor search. In International Conference on Learning Representations, 2019. URL https://api.semanticscholar.org/CorpusID:189999681
work page 2019
Show all 24 references
-
[9]
and Long, C
Gao, J. and Long, C. Rabitq: quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search. Proceedings of the ACM on Management of Data, 2 0 (3): 0 1--27, 2024
2024
-
[10]
Optimized product quantization
Ge, T., He, K., Ke, Q., and Sun, J. Optimized product quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36 0 (4): 0 744--755, 2014. doi:10.1109/TPAMI.2013.240
2014 doi
-
[11]
Unleashing graph partitioning for large-scale nearest neighbor search, 2024
Gottesbüren, L., Dhulipala, L., Jayaram, R., and Lacki, J. Unleashing graph partitioning for large-scale nearest neighbor search, 2024. URL https://arxiv.org/abs/2403.01797
2024 arXiv
-
[12]
CXL-ANNS : Software-Hardware collaborative memory disaggregation and computation for Billion-Scale approximate nearest neighbor search
Jang, J., Choi, H., Bae, H., Lee, S., Kwon, M., and Jung, M. CXL-ANNS : Software-Hardware collaborative memory disaggregation and computation for Billion-Scale approximate nearest neighbor search. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pp.\ 585--600, Bosto...
2023
-
[13]
Product quantization for nearest neighbor search
Jegou, H., Douze, M., and Schmid, C. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence, 33 0 (1): 0 117--128, 2010
2010
-
[14]
Billion-scale similarity search with gpus
Johnson, J., Douze, M., and J \'e gou, H. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7 0 (3): 0 535--547, 2019
2019
-
[15]
V., Vedurada, J., et al
Khan, S., Singh, S., Simhadri, H. V., Vedurada, J., et al. Bang: Billion-scale approximate nearest neighbor search using a single gpu. arXiv preprint arXiv:2401.11324, 2024
2024 arXiv
-
[16]
u ttler, H., Lewis, M., Yih, W.-t., Rockt \
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K \"u ttler, H., Lewis, M., Yih, W.-t., Rockt \"a schel, T., et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33: 0 9459--9474, 2020
2020
-
[17]
Malkov, Y. A. and Yashunin, D. A. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence, 42 0 (4): 0 824--836, 2018
2018
-
[18]
Cagra: Highly parallel graph construction and approximate nearest neighbor search for gpus
Ootomo, H., Naruse, A., Nolet, C., Wang, R., Feher, T., and Wang, Y. Cagra: Highly parallel graph construction and approximate nearest neighbor search for gpus. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pp.\ 4236--4247. IEEE, 2024
2024
-
[19]
Lm-diskann: Low memory footprint in disk-native dynamic graph-based ann indexing
Pan, Y., Sun, J., and Yu, H. Lm-diskann: Low memory footprint in disk-native dynamic graph-based ann indexing. In 2023 IEEE International Conference on Big Data (BigData), pp.\ 5987--5996. IEEE, 2023
2023
-
[20]
J., Devvrit, Kadekodi, R., Krishaswamy, R., and Simhadri, H
Subramanya, S. J., Devvrit, Kadekodi, R., Krishaswamy, R., and Simhadri, H. V. Diskann: Fast accurate billion-point nearest neighbor search on a single node. Advances in neural information processing Systems, 32, 2019
2019
-
[21]
Aisaq: All-in-storage anns with product quantization for dram-free information retrieval
Tatsuno, K., Miyashita, D., Ikeda, T., Ishiyama, K., Sumiyoshi, K., and Deguchi, J. Aisaq: All-in-storage anns with product quantization for dram-free information retrieval. arXiv preprint arXiv:2404.06004, 2024
2024 arXiv
-
[22]
Wang, Q. C. B. Z. H. W. M. L. C. L. Z. L. M. Y. J. Spann: Highly-efficient billion-scale approximate nearest neighbor search. In 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021
2021
-
[23]
Wang., Q. C. H. W. M. L. G. R. S. L. J. Z. J. L. C. L. L. Z. J. SPTAG : A library for fast approximate nearest neighbor search. https://github.com/Microsoft/SPTAG, 2018
2018
-
[24]
Song: Approximate nearest neighbor search on gpu
Zhao, W., Tan, S., and Li, P. Song: Approximate nearest neighbor search on gpu. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp.\ 1033--1044. IEEE, 2020
2020
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.