Pith. sign in

REVIEW 3 major objections 5 minor 20 references

kANNolo: Sweet and Smooth Approximate k-Nearest Neighbors Search

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A modular Rust library for approximate nearest neighbors reports up to 11.1x speedup over Faiss on dense data.

desk verdict kANNolo is a genuinely modular ANN library with a useful trait-based architecture, but its state-of-the-art performance claim, especially on sparse vectors, is not supported by the evaluation as written. read the letter →

arxiv 2501.06121 v2 pith:P5WVJWIQ submitted 2025-01-10 cs.IR

Leonardo Delfino , Domenico Erriquez , Silvio Martinico , Franco Maria Nardini , Cosimo Rulli , Rossano Venturini This is my paper · ORCID
classification cs.IR
keywords approximatenearestneighborsearchRustlibrarymodulararchitectureHNSWproductquantizationdenseretrievalsparsespeed-accuracytrade-off
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

kANNolo is a new approximate-nearest-neighbor (ANN) search library written in Rust, built around four composable abstractions: data arrays (dense or sparse), quantizers, query evaluators, and datasets, all connected through Rust traits. The paper's central claim is that this modularity does not cost performance: on the Sift1M and MS MARCO benchmarks, kANNolo matches or beats specialized libraries, with reported speedups up to 11.1x over Faiss on dense data and 2.1x over the best sparse-track competitors. The authors argue this makes it possible to prototype new indexes and quantizers and test them across different similarity measures and vector types without rewriting the search pipeline.

What carries the argument

The load-bearing machinery is the Rust trait system: DArray1 normalizes dense and sparse vectors behind one interface; Quantizer wraps identity or product quantization; QueryEvaluator computes distances through a unified path; Dataset ties arrays to a quantizer and an evaluator. On top of this, the index is a single implementation of Hierarchical Navigable Small World graphs (HNSW), a graph index with hierarchical links, which the paper reuses unchanged across all data types and measures. Product Quantization (splitting vectors into subspaces and encoding each subspace separately) is the one quantization method currently implemented, and the Identity quantizer allows the same code path to run without quantization.

What would settle it

Re-run the reported experiments on Sift1M and MS MARCO while sweeping each library's HNSW parameters (M, efConstruction, efSearch) and product-quantization settings at matched accuracy targets; if any competitor reaches every point on kANNolo's accuracy-QPS curve with equal or higher throughput, the state-of-the-art claim is refuted.

Watch

Extended reading notes

Core claim

The paper claims that a fully composable ANN library can combine research-friendly modularity with state-of-the-art speed-accuracy trade-offs. kANNolo achieves this by expressing data representation, quantization, distance computation, and graph indexing as interchangeable Rust traits, so one HNSW graph index doubles as the engine for dense L2 search, dense inner-product search, product-quantized search, and sparse inner-product search. The experimental section reports that kANNolo is competitive with Faiss, hnswlib, and N2 on Sift1M and outperforms them on MS MARCO dense embeddings, and that it beats the 2023 Big ANN sparse-track winners on SPLADE embeddings, with peak speedups of 11.1x and 2.1x respectively.

Load-bearing premise

The central claim depends on the benchmarks being fair and representative: the competitor libraries must be configured with equally tuned, state-of-the-art parameters, and the chosen datasets must reflect the settings each method targets, since the reported speedups are all measured against those baselines.

Editorial extensions

If this is right

  • A researcher can plug a new vector type, similarity measure, or quantizer into kANNolo and reuse the same HNSW index code, so experiments across dense, sparse, and quantized settings run in one framework.
  • If the reported speedups hold, a research-oriented library does not have to sacrifice peak performance for modularity on the tested datasets.
  • The 11.1x dense speedup over Faiss suggests that building the HNSW graph on original (unquantized) vectors while still supporting product-quantized comparison can be faster than the competitor's route of building the graph on quantized vectors.
  • Sparse retrieval with the same graph index matches or beats specialized sparse ANN implementations, which supports the view that graph-based indexes remain competitive for learned sparse embeddings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to verify whether the trait-based pipeline preserves its speed advantage when a new quantizer, such as a learned or residual quantizer, is added, since the paper only evaluates product quantization.
  • Because the paper explicitly defers inverted-index comparisons on sparse vectors, the sparse speedup claim is best read as a graph-vs-graph comparison until that stronger baseline is measured.
  • If kANNolo's architecture is as reusable as claimed, it could become a common testbed for ANN research, letting different groups share implementations of distances, quantizers, and indexes as interchangeable components rather than per-library forks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces kANNolo, a Rust ANN library designed around four trait-based components (DArray1, Quantizer, QueryEvaluator, Dataset) with HNSW as the implemented index and product quantization as the implemented quantizer. The authors claim that this composable architecture is the first to support both dense and sparse vector representations over different similarity measures, and they benchmark it against FAISS, hnswlib, N2, GrassRMA, and PyANN on Sift1M, MS MARCO dense (Star/Dragon), and MS MARCO sparse (SPLADE). The reported results show kANNolo matching or outperforming these competitors, with peak speedups of 11.1× over FAISS on dense data and 2.1× over the sparse competitors.

Significance. If the performance and modularity claims hold, kANNolo would be a valuable research-oriented ANN library: the trait-based design is a real engineering contribution, the code is publicly available, and the evaluation uses several standard public datasets. The paper also deserves credit for testing the same indexing code path across dense, sparse, and quantized settings, which directly supports the modularity claim. However, the current evidence is insufficient to support the headline 'state-of-the-art' claim, particularly for sparse vectors, because the sparse baselines are limited to other HNSW variants and the comparison with inverted indexes—the dominant approach for learned sparse representations—is explicitly deferred. The missing hyperparameter disclosure and lack of error bars further weaken the quantitative claims. The paper's significance is therefore conditional on addressing these benchmarking gaps.

major comments (3)
  1. [§3 Competitors, sparse paragraph] The abstract and §1 claim state-of-the-art performance on sparse data ('up to 2.1× speedup'), but the sparse evaluation compares only GrassRMA and PyANN, which the paper itself describes as HNSW variants 'as kANNolo.' The same paragraph states, 'We leave the comparison with inverted indexes for future work [3].' Inverted indexes are the dominant and strongest baselines for learned sparse representations such as SPLADE, so the current evidence supports only 'competitive with HNSW-based sparse libraries,' not 'state-of-the-art.' Please either add inverted-index baselines to the sparse benchmark or soften the abstract and §1 claims accordingly.
  2. [§3.1 Experimental Results / Figure 3] Figure 3 presents the central performance evidence as speed-accuracy curves without error bars, number of repetitions, or any statistical significance measure. The text also does not report the HNSW parameters (M, efConstruction, efSearch) or product-quantization parameters used for kANNolo and for each competitor. Without this information, the reported speedups—especially the 11.1× improvement over FAISS—cannot be reproduced or verified, and the reader cannot rule out configuration choices that favor kANNolo. Please provide a full configuration table, run repeated trials, and report variance or a statistical comparison of the curves.
  3. [Abstract / §1 Introduction] The abstract claims that kANNolo 'is the first ANN library that supports dense and sparse vector representations made available on top of different similarity measures.' This is an absolute novelty claim that is not substantiated by a systematic comparison or related-work survey. If the claim is retained, the paper should justify it with concrete evidence, such as a table listing existing libraries and their supported data types, similarity measures, and quantization options; otherwise it should be weakened to 'to our knowledge' or removed.
minor comments (5)
  1. [§3 Reproducibility and Hardware Details] The reproducibility section reports compiler version and CPU, but it omits index build times, memory usage, query set details, and dataset preprocessing steps; adding these would materially improve reproducibility.
  2. [Table 1] The table lists the SPLADE dataset dimensionality as '—'; please clarify the effective number of nonzeros or the vocabulary dimension used, since this affects the interpretation of the sparse results.
  3. [Figure 3] The panels use different x-axis ranges and scales (e.g., SPLADE covers only 0.90–0.97), which makes cross-dataset comparison difficult; consider consistent axes or separate labeled plots with higher resolution.
  4. [§3.1 Performance] The sentence explaining the 11.1× speedup over FAISS refers to building the HNSW graph on original rather than quantized vectors; please clarify whether the reported speedup is for search time, indexing time, or end-to-end pipeline time, as this changes the interpretation.
  5. [References] Several competitor links (GrassRMA, PyANN, N2) appear only as bare URLs; for a systems paper, full formal references with version or access date would be more appropriate.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity: all headline claims are external benchmark comparisons; the only self-citation is a non-load-bearing future-work pointer.

full rationale

kANNolo's central claims are empirical: it reports measured accuracy-versus-throughput curves on public datasets (Sift1M, MS MARCO Star/Dragon dense, SPLADE sparse) against external libraries (FAISS, hnswlib, N2, GrassRMA, PyANN). There is no fitted parameter later renamed as a prediction, no quantity defined in terms of the quantity it claims to predict, and no uniqueness theorem or ansatz imported from the authors' prior work. The only reference to the authors' own work is the future-work sentence in Section 3, 'We leave the comparison with inverted indexes for future work [3],' citing Bruch et al. (2024) by three of the present co-authors. That citation is not used to justify any measured result; it explicitly acknowledges an omitted baseline. The omission of inverted-index baselines weakens the generality of the sparse 'state-of-the-art' wording, but that is an evidence/completeness concern, not circularity. HNSW and PQ are standard methods the library implements; the 'first ANN library' claim is a feature comparison, not a derivation. The paper is therefore self-contained against external benchmarks for the speed-accuracy claims it actually plots, and the sole self-citation is not load-bearing.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard ANN algorithms and benchmark data rather than on new mathematical assumptions. The free parameters are tuning hyperparameters that are not reported, which is a reproducibility gap but not an invented entity.

free parameters (3)
  • HNSW graph construction parameters (M, efConstruction) = not reported
    These control graph quality and build cost; the paper does not state their values, but they directly affect the reported speed-accuracy trade-off.
  • HNSW search parameter (efSearch) = not reported
    Varying efSearch generates the accuracy-QPS curves in Figure 3; it is a tuning parameter, and its values are not disclosed.
  • Product quantization parameters (number of subquantizers, bits per subquantizer) = not reported
    Used in the Dragon PQ experiment; values are not disclosed, and they affect both accuracy and speed.
assumptions (3)
  • domain assumption HNSW as described in Malkov and Yashunin provides state-of-the-art ANN performance on the tested datasets.
    The paper builds kANNolo's index on the HNSW algorithm and relies on its known effectiveness rather than re-deriving it.
  • domain assumption Product quantization as described by Jegou et al. compresses vectors without excessive accuracy loss.
    The PQ component is assumed to work as in the literature; the paper does not analyze quantization error.
  • domain assumption Public benchmark datasets (Sift1M, MS MARCO) are representative of real ANN workloads.
    The paper generalizes from these datasets to ANN practice in general.

how reviews work

0 comments
Cite this review

Pith. "Pith review of kANNolo: Sweet and Smooth Approximate k-Nearest Neighbors Search." pith.science (2026). https://pith.science/paper/P5WVJWIQ

@misc{pith2026250106121,
  author       = {Pith},
  title        = {Pith review of: kANNolo: Sweet and Smooth Approximate k-Nearest Neighbors Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P5WVJWIQ}},
  note         = {Machine review of arXiv:2501.06121}
}
read the original abstract

Approximate Nearest Neighbors (ANN) search is a crucial task in several applications like recommender systems and information retrieval. Current state-of-the-art ANN libraries, although being performance-oriented, often lack modularity and ease of use. This translates into them not being fully suitable for easy prototyping and testing of research ideas, an important feature to enable. We address these limitations by introducing kANNolo, a novel research-oriented ANN library written in Rust and explicitly designed to combine usability with performance effectively. kANNolo introduces a fully composable architecture for ANN search that supports both dense and sparse vector representations. It enables researchers to seamlessly mix and match different similarity measures, vector quantization techniques (e.g., Product Quantization), and index structures (e.g., HNSW) within a single unified framework. These functionalities are managed through Rust traits, allowing shared behaviors to be handled abstractly. This abstraction ensures flexibility and facilitates an easy integration of new components. In this work, we detail the architecture of kANNolo and demonstrate that its flexibility does not compromise performance. The experimental analysis shows that kANNolo achieves state-of-the-art performance in terms of speed-accuracy trade-off while allowing fast and easy prototyping, thus making kANNolo a valuable tool for advancing ANN research. Source code available on GitHub: https://github.com/TusKANNy/kannolo.

Figures

Figures reproduced from arXiv: 2501.06121 by the authors.

Figure 1
Figure 1. Indexing with kANNolo [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Accuracy vs. queries per second. Left to right: Dense vectors, no quantization (1-3), dense vectors with product quantization (4), sparse vectors, no quantization (5). 4 Conclusions and Future Work We introduced kANNolo, a Rust-based library for approximate nearest neighbors search. It is designed to ease the prototyping and development of new ANN algorithms. Our extensive benchmarking on several public dense/sparse… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 10 canonical work pages

  1. [3]

    In: Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (2024), https://api.semanticscholar.org/CorpusID:269449081

    Bruch, S., Nardini, F.M., Rulli, C., Venturini, R.: Efficient inverted indexes for approximate retrieval over learned sparse representations. In: Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (2024), https://api.semanticscholar.org/CorpusID:269449081

    work page 2024

  2. [1]

    Information Systems 87 (02 2019), https://github.com/erikbern/ann-benchmarks

    Aumüller, M., Bernhardsson, E., Faithfull, A.: Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms. Information Systems 87 (02 2019), https://github.com/erikbern/ann-benchmarks

    work page 2019

  3. [2]

    In: Brisaboa, N.R., Pedreira, O., Zezula, P

    Boytsov, L., Naidan, B.: Engineering efficient and effective non-metric space li- brary. In: Brisaboa, N.R., Pedreira, O., Zezula, P. (eds.) Similarity Search and Ap- plications - 6th International Conference, SISAP 2013, A Coruña, Spain, October 2-4, 2013, Proceedings. Lecture Notes in Computer Science, vol. 8199, pp. 280–

    work page 2013

  4. [4]

    arXiv preprint arXiv:2401.08281 (2024)

    Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.E., Lomeli, M., Hosseini, L., Jégou, H.: The faiss library. arXiv preprint arXiv:2401.08281 (2024)

    arXiv 2024

  5. [5]

    In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Formal, T., Piwowarski, B., Clinchant, S.: Splade: Sparse lexical and expansion model for first stage ranking. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 2288–2292. SIGIR ’21, Association for Computing Machinery, New York, NY, USA(2021). https://doi.org/10.1145/3404835.3463098, h...

    arXiv 2021

  6. [6]

    https://github.com/Leslie-Chung/GrassRMA

    GrassRMA. https://github.com/Leslie-Chung/GrassRMA

    work page

  7. [7]

    In: International Conference on Machine Learning (2020),https://arxiv.org/abs/1908.10396

    Guo, R., Sun, P., Lindgren, E., Geng, Q., Simcha, D., Chern, F., Kumar, S.: Accel- erating large-scale inference with anisotropic vector quantization. In: International Conference on Machine Learning (2020),https://arxiv.org/abs/1908.10396

    arXiv 2020

  8. [8]

    IEEE transactions on pattern analysis and machine intelligence33, 117–28 (01 2011)

    Jégou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence33, 117–28 (01 2011). https://doi.org/10.1109/TPAMI.2010.57

    doi:10.1109/tpami.2010.57 2011

Show all 20 references
  1. [9]

    CoRR abs/2302.07452 (2023)

    Lin, S., Asai, A., Li, M., Oguz, B., Lin, J., Mehdad, Y., Yih, W., Chen, X.: How to train your DRAGON: diverse augmentation towards generalizable dense retrieval. CoRR abs/2302.07452 (2023). https://doi.org/10.48550/ARXIV.2302.07452, https://doi.org/10.48550/arXiv.2302.07452

    2023 doi arXiv

  2. [10]

    In: Proceedings of the Seventh IEEE International Conference on Computer Vision

    Lowe, D.: Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision. vol. 2, pp. 1150–1157 vol.2 (1999).https://doi.org/10.1109/ICCV.1999.790410

    1999

  3. [11]

    IEEE Trans

    Malkov, Y.A., Yashunin, D.A.: Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Trans. Pattern Anal. Mach. Intell.42(4), 824–836 (apr 2020).https://doi.org/10.1109/TPAMI.2018. 2889473, https://github.com/nmslib/hnswlib

    2020 doi

  4. [12]

    https://github.com/kakao/n2

    N2. https://github.com/kakao/n2

  5. [13]

    In: Besold, T.R., Bordes, A., d’Avila Garcez, A.S., Wayne, G

    Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., Deng, L.: MS MARCO: A human generated machine reading comprehension dataset. In: Besold, T.R., Bordes, A., d’Avila Garcez, A.S., Wayne, G. (eds.) Proceed- ings of the Workshop on Cognitive Computation: Int...

    2016

  6. [14]

    https://github.com/veaaaab/pyanns

    pyanns. https://github.com/veaaaab/pyanns

  7. [15]

    Simhadri, H.V., Aumüller, M., Ingber, A., Douze, M., Williams, G., Manohar, M.D., Baranchuk, D., Liberty, E., Liu, F., Landrum, B., Karjikar, M., Dhulipala, L., Chen, M., Chen, Y., Ma, R., Zhang, K., Cai, Y., Shi, J., Chen, Y., Zheng, W., Wan, Z., Yin, J., Huang, B.: Results o...

    2024

  8. [16]

    Simhadri, H.V., Krishnaswamy, R., Srinivasa, G., Subramanya, S.J., Antonije- vic, A., Pryce, D., Kaczynski, D., Williams, S., Gollapudi, S., Sivashankar, V., Karia, N., Singh, A., Jaiswal, S., Mahapatro, N., Adams, P., Tower, B., Patel, Y.: DiskANN: Graph-structured Indices fo...

    2023

  9. [17]

    In: Neural Information Processing Systems (2023), https://arxiv.org/abs/2404.00774

    Sun, P., Simcha, D., Dopson, D., Guo, R., Kumar, S.: Soar: Improved indexing for approximate nearest neighbor search. In: Neural Information Processing Systems (2023), https://arxiv.org/abs/2404.00774

    2023 arXiv

  10. [18]

    In: Proceedings of the 44th Inter- national ACM SIGIR Conference on Research and Development in Informa- tion Retrieval

    Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., Ma, S.: Optimizing dense re- trieval model training with hard negatives. In: Proceedings of the 44th Inter- national ACM SIGIR Conference on Research and Development in Informa- tion Retrieval. p. 1503–1512. SIGIR ’21, Associatio...

    2021

  11. [293]

    Springer (2013).https://doi.org/10.1007/978-3-642-41062-8_28 , https: //doi.org/10.1007/978-3-642-41062-8_28

    2013 doi

  12. [2016]

    CEUR Workshop Proceedings, vol. 1773. CEUR-WS.org (2016), https: //ceur-ws.org/Vol-1773/CoCoNIPS_2016_paper9.pdf kANNolo: Sweet and Smooth Approximatek-Nearest Neighbors Search 7

    2016

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.