Pith. sign in

REVIEW 4 major objections 4 minor 24 references

DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read The paper argues that a single distributed graph index can beat partition-and-route architectures for web-scale approximate nearest-neighbor search.

desk verdict A credible production ANN serving architecture with a genuine 50B-vector deployment, but the 6x efficiency headline is only as strong as the single clustered-partitioning baseline it is measured against. read the letter →

arxiv 2509.06046 v1 pith:6727ZJZH submitted 2025-09-07 cs.DC cs.DScs.IR

classification cs.DCcs.DScs.IR
keywords approximatenearestneighborsearchdistributedvectorgraphindexDISKANNkey-valuestorenear-datacomputationclusteredpartitioningweb-scaleretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most web-scale vector search systems split the corpus into partitions, build one index per partition, and route each query to a subset of partitions; the paper argues this is not the only efficient path. It presents DISTRIBUTEDANN, which keeps one logical DISKANN graph over the whole corpus and serves it from a distributed key-value store treated as a shared disk. On a 50-billion-vector slice of a production web index across over a thousand machines, the system reports 26 ms median query latency and more than 100,000 queries per second, roughly 6x the throughput of a clustered-partitioning baseline at similar recall. The system has replaced the prior partitioned service in Bing search, and the paper attributes the gain to three layout changes: duplicating compressed vectors into graph nodes, adding an in-memory head index over the graph's top layers, and scoring nodes near the data. The trade is more SSD space (780 TiB vs 270 TiB) and higher latency (26 ms vs 16 ms median), which the authors judge favourable for their workloads.

What carries the argument

The load-bearing object is the single logical DISKANN graph served from a distributed key-value store, adapted by three mechanisms: (1) compressed vectors are duplicated into every graph node that references them, collapsing the many lookups of a beam-search iteration into one batched remote read; (2) an in-memory head index is built over the graph's top layers, giving a network-hop-free seeding step; (3) a node-scoring service runs on each storage host, computing distances locally and returning compact scores and candidate IDs. Together these turn a graph built for one machine's SSD into a distributed structure whose cost per query grows with graph hops, not with the number of partitions.

What would settle it

Run the same 50-billion-vector slice with the advanced graph-partitioning schemes the paper names as future work in Section 4.4, at the same machine footprint and the same 15k-QPS resource bound, and compare recall@5 and recall@200 at that throughput; if such a partitioner matches DISTRIBUTEDANN's recall or reaches comparable throughput at equal recall, the claimed 6x advantage is falsified.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that state-of-the-art performance on very-large-scale ANN search can be achieved through a single large logical index stored in a distributed key-value store. The authors take a DISKANN graph—normally designed for one machine's local SSD—and adapt it for network access by duplicating each vector's compressed representation into all graph nodes that point to it, so one beam-search hop needs only one batched read. They add a sharded in-memory head index over the graph's top layers so search starts from good candidates without paying network hops for the first iterations. They also push node scoring into a service running on each key-value store host, which re

Load-bearing premise

The 6x claim rests on the clustered-partitioning baseline being representative; the paper has not compared against the strongest graph-partitioning schemes and says these may close the gap.

Editorial extensions

If this is right

  • Sublinear scaling: as the corpus grows, query cost is tied to graph hops rather than the number of partitions, so the cost curve flattens at web scale.
  • Flexible IO allocation: a unified graph lets a query spend its IO budget where the nearest neighbours actually live, yielding higher recall at the same IO per query.
  • Throughput scaling by sharding: adding hosts to the key-value store adds IOPS and CPU without increasing partitions searched per query, so the system is bound by SSD space rather than IO.
  • Graceful degradation: partial failures of node-scoring requests cause recall loss roughly proportional to the failure rate, so operators can safely timeout stragglers.
  • Head-index bottleneck: the CPU-bound head index is the current limiter; adding head replicas raises QPS without adding machines because total SSD space remains the bounding resource.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the advanced graph-partitioning baselines named in Section 4.4 close most of the gap, the durable contribution shifts from '6x faster' to 'one graph avoids the operational complexity of load-balancing many partitions under shifting traffic.'
  • The space amplification from duplicated compressed vectors is the main economic cost; packing several nearby full-dimensional vectors into one graph node, which the paper flags as future work, is a concrete experiment that could reduce that cost substantially.
  • If the head-index CPU bound is relieved by a GPU or kernel-bypass networking, the latency disadvantage relative to partitioned systems could shrink further without changing the graph layout.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents DISTRIBUTEDANN, a distributed ANN serving system that stores a single large DISKANN graph in a distributed key-value store, with three main modifications: duplicating compressed vector representations into graph nodes, maintaining an in-memory 'head index' over the top layers of the graph, and performing near-data scoring at each key-value host. The system is reported to serve a 50-billion-vector slice of the Bing web index at over 100,000 QPS with 26ms median latency, claimed to be 6x more efficient than a clustered-partitioning and routing baseline. The paper also describes a graph-stitching construction method, reliability experiments under partial node-scoring failures, and a deployment experience in Microsoft Bing.

Significance. If validated, the result is significant: it demonstrates that a single logical ANN graph can be scaled across thousands of machines in a shared-nothing cluster, avoiding the linear cost growth of partitioned-and-routed serving while reusing commodity key-value store infrastructure. The paper's principal strengths are its production scale, concrete measurements on a 50B-vector workload, the clear articulation of bandwidth and computation tradeoffs (Eq. 1-2), and an explicit discussion of failure modes. However, the central efficiency claim rests on a single baseline comparison and several experimental details are underspecified, so the contribution is currently conditional rather than fully established.

major comments (4)
  1. [§4, Table 1] The headline '6x more efficient' (Abstract and §4) rests on a single comparison in Table 1. DISTRIBUTEDANN is evaluated with H=5, BW=128, R=72, 640 IO/query, recall@200=71.9, and >100k QPS; the clustered-partitioning baseline searches 40 of 203 partitions with I=120 per partition, 4800 IO/query, recall@200=67.4, and ~15k QPS. The throughput ratio therefore conflates the serving architecture, the IO budget, and the achieved recall. The statement that parameters are chosen for 'similar footprint (by bounding resource) at 15k QPS' is not accompanied by a quantitative method for bounding. To support the 6x claim, the comparison should be at matched recall/IO, or the baseline should be demonstrated to be a strong representative of the state of the art.
  2. [§4.4] The paper concedes that advanced partitioning schemes (Dong et al. 2019; Gottesbüren et al. 2024) reduce cross-partition edges and 'may be preferable in very latency-constrained scenarios', and states that 'further work is needed' to compare them empirically. This directly limits the central claim that DISTRIBUTEDANN is '6x more efficient than existing partitioning and routing strategies'. Either include a stronger partitioning baseline, or explicitly restrict the claim to the specific clustered partitioner implemented here and to throughput-oriented serving.
  3. [§4, recall evaluation] Recall ground truth is not specified. For the 50B-vector slice, there is no description of how true nearest neighbors were computed (e.g., exhaustive search, ground-truth set size, sampling procedure). Without this, the recall values in Table 1 and Figure 4 cannot be reproduced or interpreted. Additionally, all throughput and latency numbers appear to be single-point measurements taken in a production environment with mixed workloads; the paper should report variance or repeated trials to support the reported QPS and latency figures.
  4. [§5, Conclusion] The conclusion states that DISTRIBUTEDANN 'achieves sublinear scaling of ANN search on very large datasets', but no experiment varies the dataset size or the number of machines. The evidence is one 50B-vector slice at a single configuration. While the log-index complexity argument is analytically plausible, the empirical claim of sublinear scaling across scales is not demonstrated. Please add a scaling study or soften the conclusion to a single-scale demonstration.
minor comments (4)
  1. [Abstract / §1] The abstract says '50 billion vector graph index' while §1 says 'fifty-billion vector subset of a web search dataset'. Clarify whether the evaluated index is a slice of a larger corpus and whether the multiplicity of partition replicas is included in the 50B count.
  2. [§2.2, Eq. (1)-(2)] The formulas for space amplification and bandwidth savings would benefit from clearer parentheses and explicit definitions of d, d_OPQ, and R. As written, the numerator and denominator are ambiguous.
  3. [§4.1] The sentence 'at 3 replicas becomes CPU-bound' is unclear: does this refer to the head index replicas or the node-scoring service replicas? Specify which replica count is being varied and what '3 replicas' corresponds to in Table 1.
  4. [Figure 4] The x-axis is labeled 'IO per Query (log scale)' but the figure caption does not state whether the plotted IO counts include the head-index search or only the graph-node reads. Please clarify the IO accounting across both systems.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's headline numbers are empirical measurements, and the self-citations used are to externally validated building blocks, not to definitions or fitted targets.

full rationale

The paper's central claims—26ms median latency, >100k QPS, 6x efficiency, and recall improvements—are reported experimental results from a deployed system, not predictions derived from equations containing those targets. The only equations, Eqs. (1) and (2), compute space amplification and bandwidth savings from the chosen layout parameters; these are algebraic consequences of the design, not circular validations of the headline numbers. The self-citations to DISKANN and SPANN (Subramanya et al., 2019; Wang, 2021) are used as building blocks: DISKANN provides the single-node graph index and SPANN the graph-stitching technique. These are prior, externally published systems with their own empirical support, and the paper does not invoke a uniqueness theorem or a fitted parameter to force its conclusions. Section 4.4 explicitly acknowledges that advanced partitioning schemes may narrow or close the efficiency gap and calls for further comparison; this is a limitation regarding the strength of the baseline, not a circularity. The '6x' claim depends on the representativeness of the clustered-partitioning baseline, but that is a benchmarking validity concern, not a self-referential derivation. Overall, no step in the paper reduces by construction to its own inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claims rest on four assumptions: stitched-graph quality suffices, random sharding yields predictable scaling, recall ground truth is valid, and the head index provides reachable entry points. The free parameters are the tuned system and baseline configurations that the performance comparison depends on; the invented entities are new system components whose effectiveness is shown only through the paper's own evaluation.

free parameters (4)
  • Graph degree R (DISTRIBUTEDANN) = 72 (truncated from 106)
    Chosen to reduce KV-store space; affects the recall/IO tradeoff reported in Figure 4.
  • Head index size = 2.5 billion vectors
    Chosen as a memory/compute tradeoff; the paper notes it becomes CPU-bound at 3 replicas.
  • Beam search parameters (BW, H) = 128, 5
    Reported in Section 4 as the DISTRIBUTEDANN operating point on the recall/IO frontier.
  • Baseline partition routing (top-N partitions) = 40 of 203
    The conventional system's parameters (N=40, I=120, BW=6, R=106) were selected so both systems have similar footprint at 15k QPS; the 6x claim depends on this choice.
assumptions (4)
  • domain assumption A stitched graph built from clustered partitions preserves enough graph quality to support beam search recall comparable to an incrementally built graph.
    Section 3 states the quality is 'lower than one built entirely incrementally, but is sufficient to get good results'; if this fails, the recall numbers would not hold.
  • domain assumption Random sharding of the KV store yields predictable traffic and allows throughput scaling by adding hosts without changing routing.
    Sections 2.4 and 4.1 rely on this to claim efficient/sublinear scaling; the paper cites Dynamo-style availability but does not prove the load-balancing claim empirically beyond the reported production numbers.
  • domain assumption Recall on sampled web queries is measured against a valid ground truth, the construction of which is not described.
    Section 4 reports recall@5 and recall@200 but does not specify how ground truth is obtained for a 50B-vector production index; this is a measurement-methodology assumption.
  • domain assumption A head index built by BFS over the top layers of the graph provides entry points that reach the rest of the graph within H hops.
    Section 2.2 introduces the head index but does not prove reachability; it relies on DISKANN graph properties and is validated only by the end-to-end evaluation.
invented entities (2)
  • Head index
    purpose: A sharded in-memory ANN index over the top C vectors of the graph, searched first to obtain starting points for beam search and avoid network hops in early iterations.
    Its benefit is demonstrated only through the paper's own evaluation; no external validation or separate ablation isolates its contribution.
  • Near-data node scoring service
    purpose: Runs on each KV host to score graph nodes and their OPQ candidates locally, returning scores instead of node data to reduce network bandwidth and parallelize compute.
    Validated only through in-paper experiments; the paper does not provide a standalone benchmark or ablation for this component.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers." pith.science (2026). https://pith.science/paper/6727ZJZH

@misc{pith2026250906046,
  author       = {Pith},
  title        = {Pith review of: DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6727ZJZH}},
  note         = {Machine review of arXiv:2509.06046}
}
read the original abstract

We present DISTRIBUTEDANN, a distributed vector search service that makes it possible to search over a single 50 billion vector graph index spread across over a thousand machines that offers 26ms median query latency and processes over 100,000 queries per second. This is 6x more efficient than existing partitioning and routing strategies that route the vector query to a subset of partitions in a scale out vector search system. DISTRIBUTEDANN is built using two well-understood components: a distributed key-value store and an in-memory ANN index. DISTRIBUTEDANN has replaced conventional scale-out architectures for serving the Bing search engine, and we share our experience from making this transition.

Figures

Figures reproduced from arXiv: 2509.06046 by the authors.

Figure 1
Figure 1. High-level architecture comparison between a conventional system using clustered partitioning and DISTRIBUTEDANN. provements in recall@5 and recall@200 respectively. 2. DISTRIBUTEDANN Existing ANN indices like DISKANN are optimized for tiered storage by reducing the number of round-trips to SSD. Therefore, DISKANN is well suited for adaptation to a dis￾tributed serving environment, since the main drawback of accessi… view at source ↗
Figure 3
Figure 3. IO in each cluster for a single query, as served by DIS￾TRIBUTEDANN and a conventional clustered partitioning index. DISTRIBUTEDANN is able to implement a much more flexible traversal strategy, improving efficiency. traditional clustered partitioning approach (with roughly 200 million vectors per partition) on one slice of the in￾dex. Because of the graph stitching approach described in Section 3, we are able to ing… view at source ↗
Figure 2
Figure 2. A visual depiction of the unified graph construction pro￾cess. Partitions are represented by colored regions, and neighbor￾hoods by dark shading. When points exist in multiple partitions, their neighborhoods will be merged by taking the union of neigh￾bor lists, yielding a unified graph. 4. Evaluation The Bing web index is composed of multiple independent slices to allow portions of the index to be updated atomi￾cal… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Optimal Recall/IO frontier for indexing 50 billion vectors. Grid search of parameters, DISTRIBUTEDANN: H from 4 to 8, BW = 32i for i from 3 to 6. Clustered Partitioning: selected clusters N = {20, 25, 30, 40, 50, 60}, IO per cluster M = 32i for i from 2 to 6. graph. Be…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 18 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    and Indyk, P

    Andoni, A. and Indyk, P. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Communications of the ACM, 51 0 (1): 0 117--122, 2008

  3. [3]

    Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms, 2018

    Aumüller, M., Bernhardsson, E., and Faithfull, A. Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms, 2018. URL https://arxiv.org/abs/1807.05614

  4. [4]

    and Lempitsky, V

    Babenko, A. and Lempitsky, V. The inverted multi-index. IEEE transactions on pattern analysis and machine intelligence, 37 0 (6): 0 1247--1260, 2014

  5. [5]

    Big-ann benchmarks: Neurips 2023

    Big-ANN Benchmarks . Big-ann benchmarks: Neurips 2023. https://big-ann-benchmarks.com/neurips23.html, 2023. Accessed: 2025-01-04

  6. [6]

    and Barroso, L

    Dean, J. and Barroso, L. A. The tail at scale. Communications of the ACM, 56 0 (2): 0 74--80, 2013

  7. [7]

    Dynamo: Amazon's highly available key-value store

    DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., and Vogels, W. Dynamo: Amazon's highly available key-value store. ACM SIGOPS operating systems review, 41 0 (6): 0 205--220, 2007

  8. [8]

    P., and Wagner, T

    Dong, Y., Indyk, P., Razenshteyn, I. P., and Wagner, T. Learning space partitions for nearest neighbor search. In International Conference on Learning Representations, 2019. URL https://api.semanticscholar.org/CorpusID:189999681

Show all 24 references
  1. [9]

    and Long, C

    Gao, J. and Long, C. Rabitq: quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search. Proceedings of the ACM on Management of Data, 2 0 (3): 0 1--27, 2024

  2. [10]

    Optimized product quantization

    Ge, T., He, K., Ke, Q., and Sun, J. Optimized product quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36 0 (4): 0 744--755, 2014. doi:10.1109/TPAMI.2013.240

  3. [11]

    Unleashing graph partitioning for large-scale nearest neighbor search, 2024

    Gottesbüren, L., Dhulipala, L., Jayaram, R., and Lacki, J. Unleashing graph partitioning for large-scale nearest neighbor search, 2024. URL https://arxiv.org/abs/2403.01797

  4. [12]

    CXL-ANNS : Software-Hardware collaborative memory disaggregation and computation for Billion-Scale approximate nearest neighbor search

    Jang, J., Choi, H., Bae, H., Lee, S., Kwon, M., and Jung, M. CXL-ANNS : Software-Hardware collaborative memory disaggregation and computation for Billion-Scale approximate nearest neighbor search. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pp.\ 585--600, Bosto...

  5. [13]

    Product quantization for nearest neighbor search

    Jegou, H., Douze, M., and Schmid, C. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence, 33 0 (1): 0 117--128, 2010

  6. [14]

    Billion-scale similarity search with gpus

    Johnson, J., Douze, M., and J \'e gou, H. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7 0 (3): 0 535--547, 2019

  7. [15]

    V., Vedurada, J., et al

    Khan, S., Singh, S., Simhadri, H. V., Vedurada, J., et al. Bang: Billion-scale approximate nearest neighbor search using a single gpu. arXiv preprint arXiv:2401.11324, 2024

  8. [16]

    u ttler, H., Lewis, M., Yih, W.-t., Rockt \

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K \"u ttler, H., Lewis, M., Yih, W.-t., Rockt \"a schel, T., et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33: 0 9459--9474, 2020

  9. [17]

    Malkov, Y. A. and Yashunin, D. A. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence, 42 0 (4): 0 824--836, 2018

  10. [18]

    Cagra: Highly parallel graph construction and approximate nearest neighbor search for gpus

    Ootomo, H., Naruse, A., Nolet, C., Wang, R., Feher, T., and Wang, Y. Cagra: Highly parallel graph construction and approximate nearest neighbor search for gpus. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pp.\ 4236--4247. IEEE, 2024

  11. [19]

    Lm-diskann: Low memory footprint in disk-native dynamic graph-based ann indexing

    Pan, Y., Sun, J., and Yu, H. Lm-diskann: Low memory footprint in disk-native dynamic graph-based ann indexing. In 2023 IEEE International Conference on Big Data (BigData), pp.\ 5987--5996. IEEE, 2023

  12. [20]

    J., Devvrit, Kadekodi, R., Krishaswamy, R., and Simhadri, H

    Subramanya, S. J., Devvrit, Kadekodi, R., Krishaswamy, R., and Simhadri, H. V. Diskann: Fast accurate billion-point nearest neighbor search on a single node. Advances in neural information processing Systems, 32, 2019

  13. [21]

    Aisaq: All-in-storage anns with product quantization for dram-free information retrieval

    Tatsuno, K., Miyashita, D., Ikeda, T., Ishiyama, K., Sumiyoshi, K., and Deguchi, J. Aisaq: All-in-storage anns with product quantization for dram-free information retrieval. arXiv preprint arXiv:2404.06004, 2024

  14. [22]

    Wang, Q. C. B. Z. H. W. M. L. C. L. Z. L. M. Y. J. Spann: Highly-efficient billion-scale approximate nearest neighbor search. In 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021

  15. [23]

    Wang., Q. C. H. W. M. L. G. R. S. L. J. Z. J. L. C. L. L. Z. J. SPTAG : A library for fast approximate nearest neighbor search. https://github.com/Microsoft/SPTAG, 2018

  16. [24]

    Song: Approximate nearest neighbor search on gpu

    Zhao, W., Tan, S., and Li, P. Song: Approximate nearest neighbor search on gpu. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp.\ 1033--1044. IEEE, 2020

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.