REVIEW 3 major objections 5 minor 1 cited by
K-Nearest Neighbor Approximation Via the Friend-of-a-Friend Principle
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The friend-of-a-friend heuristic provably fails to beat quadratic time on generic metric rankings.
desk verdict A real first attempt at a theory for nearest-neighbor descent, with an honest negative caveat and a positive proof that currently has a load-bearing gap. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two constructions carry the argument. The first is the map $\phi$ from linear orders on the $\binom{n}{2}$ inter-point distances to concordant ranking systems: each point's ranking is read off as the order of the pairs containing it. Combined with Lemma 5.9, this makes a generic CRS a setting where one point's ranking carries no information about any other point's ranking, which is exactly the condition that defeats friend-of-a-friend. The second is the acceptance-sampling step in the second neighbor range query: when two vertices propose an edge because they share a neighbor, the edge is kept with probability $g(r_t,r_{t-1})/\nu_{r_{t-1}}(v,v')$, the ratio of the minimal intersection volume of two balls to their actual intersection volume. This removes the dependence of the common-neighbor Poisson mean on the inter-vertex distance, so the updated graph samples uniformly within a shrinking ball, giving a radius update equation whose solutions decay geometrically. The paper also proves a supporting expander result (Proposition A.1) showing that the initial random friend graph has diameter $O(\log n)$.
What would settle it
Take $n$ points uniformly at random in $[0,1]^d$ with Euclidean distance, run scheduled pointwise NND with fixed $K$, and record the first round at which a fixed point's friend set contains $\lceil K/2\rceil$ of its true K-nearest neighbors. If this average round count grows like $o(n^2/K^2)$ for large $n$, then the independence-based argument of Proposition 5.8 does not describe this natural metric family; if it grows like $n^2/K^2$, the generic-CRS model would be vindicated as a worst-case description.
Extended reading notes
Core claim
The central claim is that ordinal information alone, the rankings each point has over the others, cannot rescue NND from quadratic behavior. The paper defines a concordant ranking system (CRS) as one where the linear orders each point holds on its incident pairs extend to a single partial order on all $\binom{n}{2}$ pairs; Lemma 5.5 proves this is equivalent to the ranking system being induced by a metric. It calls a CRS generic if it is $\phi(\preceq)$ for a uniformly random linear order $\preceq$ on the inter-point distances. Proposition 5.8 then shows that, running scheduled pointwise NND on a generic CRS, the expected number of rounds before a fixed point $x$ has at least $\lceil K/2\rceil$ of its true K-nearest neighbors is at least about $n^2/(2K^2)$. The mechanism is Lemma 5.9: for a uniform random linear order, the ranking of pairs touching $x$ is independent of the ranking of all other pairs, so every "friend of a friend" is, from $x$'s perspective, a uniformly random vertex. By contrast, Theorem 6.4 and Corollary 6.5 show that the second neighbor range query algorithm, which uses distances rather than ranks, runs in $O(\log n)$ rounds and $O(n\log n)$ work on the $d$-dimensional torus when $K>2d$.
Load-bearing premise
The negative result depends on modeling a "generic metric space" by a uniformly random linear ordering of all inter-point distances; the authors explicitly note this model may be biased toward rankings on which NND has quadratic complexity, so the conclusion need not extend to real metric data that is not drawn from this model.
Editorial extensions
If this is right
- NND has no subquadratic worst-case guarantee even when similarity rankings come from a metric; for generic concordant rankings, a fixed point needs about $n^2/(2K^2)$ rounds to meet half of its true neighbors.
- The friend-of-a-friend principle is not self-justifying: random initialization and the small expander diameter of the initial friend graph are not enough to make NND succeed.
- A ranking-only NND cannot be recommended solely on the strength of metricity; extra structure, such as low intrinsic dimension or correlation between nearby rankings, is needed.
- For distance-query settings, the second neighbor range query gives a provable $O(n\log n)$ work bound on the $d$-dimensional torus when $K>2d$, so the failure is specific to ordinal-only information, not to the general idea of using common neighbors.
Reading between the lines
- If real high-dimensional data sets resemble the generic-CRS model even partially, NND's observed successes likely come from ranking correlations that the uniform-order model excludes; a useful diagnostic would be to measure, for each pair of mutual friends $x,y$, the rank correlation between $r_x$ and $r_y$ restricted to the union of their friend sets.
- The independence lemma suggests a natural stress test for NND on any data set: shuffle the distances through the ranking oracle while preserving the metric, and see whether the round count moves toward $n^2/(2K^2)$.
- The acceptance-sampling idea of 2NRQ might generalize to a wider family of metric spaces with invariant measures; testing it on spheres or Euclidean balls would indicate whether the $O(\log n)$ guarantee is a torus artifact.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the friend-of-a-friend (FOF) heuristic for approximate K-nearest-neighbor graph construction. It formalizes ranking systems and concordant ranking systems (CRS), proves that concordancy is equivalent to metrizability, and introduces a 'generic' CRS model obtained as the image φ(≼) of a uniformly random linear order on the inter-point distances. Proposition 5.8 claims that scheduled pointwise NND requires at least about n^2/(2K^2) expected rounds for a fixed vertex to collect half of its true K-nearest neighbors. The paper then proposes a range-query variant, 2NRQ, and claims in Proposition 6.3, Theorem 6.4, and Corollary 6.5 that on a homogeneous Poisson process on the d-dimensional torus with K>2d, 2NRQ runs in O(log n) rounds and O(n K^2 log n) work. The appendix proves an expander/diameter bound for the initial random graph. The abstract carefully limits the negative claim to rankings arising from a generic linear order, while the body at times states the stronger conclusion that FOF fails on almost all metric spaces.
Significance. If fully established, the negative result would explain a real phenomenon: pure ranking-based FOF can fail to beat exhaustive search on generic metric-derived rankings. The positive result would give the first rigorous subquadratic analysis of an FOF-type algorithm, even though 2NRQ is a theoretical variant rather than a practical method. The conceptual apparatus of the paper—concordant ranking systems, the map φ from linear orders to CRSes, and the equivalent metrics graph—is interesting and likely to be reused. The paper is free of fitted parameters; the claimed O(log n) bound is an explicit function of n, K, d, and β. The appendix's expander argument is a self-contained contribution. The authors are commendably honest about limitations, including that the generic-CRS model may be biased toward bad instances and that pure NND remains unanalyzed. However, as written, neither of the two central claims is fully proven: the negative result is an expected-value heuristic, and the positive result relies on an independence assumption that the 2NRQ update itself destroys.
major comments (3)
- [Section 5.3, Proposition 5.8] The lower bound is an expected-value heuristic, not a rigorous complexity statement. The proof asserts that each newly discovered point is uniformly random from x's perspective, and concludes that x 'expects to have to meet about n/2 points, over at least n/(2K^2) relevant rounds.' Lemma 5.9 gives independence of the restricted linear orders only at the initial construction of the ranking system; it does not justify uniform randomness of newly discovered points conditional on the entire history of past friend sets and updates. Moreover, 'at least about n^2/(2K^2)' is not a precise mathematical statement. Since this proposition carries the abstract's claim that the basic algorithm fails to achieve subquadratic complexity on generic metric-derived rankings, the negative result needs either a rigorous high-probability treatment or an explicitly weakened statement.
- [Section 6.5, Proposition 6.3 and Eq. (14)] The induction assumes that, for a fixed pair (v,v'), the number of common neighbors in E_{t-1} is Poisson with mean n θ_{t-1}^2 ν_{r_{t-1}}(v,v')/λ(X). This would require the indicators 1_{vv''∈E_{t-1}} and 1_{v'v''∈E_{t-1}} to be conditionally independent across v'' given the Poisson configuration. The 2NRQ update of Section 6.3.2 adds an edge vv' whenever E_{t-1} contains a common neighbor of v and v', so already at t=1 the indicators 1_{vv'∈E_1} and 1_{vv''∈E_1} are correlated: a single common neighbor of v, v', and v'' can generate both edges. Consequently Eq. (11), Eq. (14), and the radius update claimed in Proposition 6.3 are not derived, and Theorem 6.4 and Corollary 6.5 inherit the gap. The paper must either prove an approximate independence or Poisson-approximation statement that survives the update, or modify the algorithm so that conditional independence is maintained by construction.
- [Sections 5.2 and 7] The 'generic CRS' model is the image of a uniformly random linear order on the pair set, not a uniform random CRS and not a uniform random metric space. The paper itself concedes in Section 7: 'It is possible that our model of a generic CRS is biased in favor of those for which NND has O(n^2) complexity.' Therefore the claim in the introduction to Section 5 that 'in almost all metric spaces FOF fails to help' is not justified by Proposition 5.8. The negative claim should either be stated only for the φ(≼) model, or the authors should compare the φ-image distribution with a uniform CRS distribution—for example, using the highly variable linear-extension counts computed in Section 5.5—to support a quasi-uniformity statement.
minor comments (5)
- [Section 5.3, Proposition 5.8] Please replace 'at least about n^2/(2K^2)' with a precise inequality, such as an explicit lower bound on the expected number of rounds or a high-probability tail bound.
- [Appendix A.3] Proposition A.3 cites 'Lemma A.1', but the statement referenced is Proposition A.1; please fix the cross-reference.
- [Section 7] There is a typo in the first sentence of the conclusions: 'simiar' should be 'similar'.
- [Section 6.6.2, Table] It would help readers if the table or its caption stated explicitly that θ_t values below 10^{-4} are omitted, and if the text explained why the ninth round would drop the average degree below K.
- [Section 4.3] The heuristic statement that x 'expects to have to meet about n/2 points' is informal and propagates into the proof of Proposition 5.8; a formal calculation or a pointer to one would strengthen the paper.
Circularity Check
No significant circularity: both the negative and positive results are derived from explicit probabilistic models, and the paper's self-citations are not load-bearing.
full rationale
The paper's central derivations are self-contained and do not reduce to their own inputs by construction. Proposition 5.8 follows from Lemma 5.9, the independence of restrictions of a uniformly random linear order to disjoint subsets, together with the stated scheduled pointwise NND update rule; the expected-rounds lower bound is a genuine calculation about the algorithm's behavior, not a restatement of the model. The 'generic CRS' notion is explicitly defined as the image phi(preceq) of a uniformly random linear order, and Section 7 explicitly acknowledges that this model may be biased toward CRSes with quadratic NND complexity; that is an honest limitation, not a circular derivation. The positive 2NRQ result in Section 6 proves a sampling invariant by induction from a stated base graph, and the acceptance-sampling rate f_t is deliberately constructed so that the edge-proposal rate does not depend on distance; this is algorithm construction intended to realize a stated goal, not a fitted parameter later relabeled as a prediction. The paper's self-citations, [7] and [8], report experiments and software; they appear in the abstract and in motivational remarks, but the rigorous claims are proved in the paper using Poisson-process calculations and the explicit torus geometry. The abstract's phrase that run times in a test case fit an O(n K^2 log n) pattern is clearly an empirical report, and the paper separately says the bound is rigorously justified for a similar algorithm in Section 6. Thus no prediction is forced by a fitting step, no load-bearing argument depends on an unverified self-citation, and no definition silently contains the theorem it is used to prove. The skeptic's concern about Proposition 6.3, that the common-neighbor update may create edge dependencies that violate the Poisson assumption, is a potential proof gap and a correctness risk, but it is not a circularity: an unproven independence claim is not the same as the theorem being equivalent to its input by construction.
Assumptions & free parameters
assumptions (4)
- ad hoc to paper The set of rankings induced by a uniformly random linear order on the pair set is a suitable model of a 'generic' metric space (Section 5.2).
- ad hoc to paper At each round of 2NRQ, the neighbors of each vertex are an independent uniform sample from its ball, so that counts of common neighbors are Poisson with the stated mean (Proposition 6.3).
- domain assumption The metric space supports a measure for which all balls of a given radius have equal measure, and the specific case is the torus with l-infinity norm (Sections 6.2, 6.4.1).
- standard math Lemma 5.9: restrictions of a uniformly random linear order to disjoint subsets are independent (stated without proof in Section 5.3).
Cite this review
Pith. "Pith review of K-Nearest Neighbor Approximation Via the Friend-of-a-Friend Principle." pith.science (2026). https://pith.science/paper/YBTSQFHE
@misc{pith2026190807645,
author = {Pith},
title = {Pith review of: K-Nearest Neighbor Approximation Via the Friend-of-a-Friend Principle},
year = {2026},
howpublished = {\url{https://pith.science/paper/YBTSQFHE}},
note = {Machine review of arXiv:1908.07645}
}
abstract
Suppose $V$ is an $n$-element set where for each $x \in V$, the elements of $V \setminus \{x\}$ are ranked by their similarity to $x$. The $K$-nearest neighbor graph is a directed graph including an arc from each $x$ to the $K$ points of $V \setminus \{x\}$ most similar to $x$. Constructive approximation to this graph using far fewer than $n^2$ comparisons is important for the analysis of large high-dimensional data sets. $K$-Nearest Neighbor Descent is a parameter-free heuristic where a sequence of graph approximations is constructed, in which second neighbors in one approximation are proposed as neighbors in the next. Run times in a test case fit an $O(n K^2 \log{n})$ pattern. This bound is rigorously justified for a similar algorithm, using range queries, when applied to a homogeneous Poisson process in suitable dimension. However the basic algorithm fails to achieve subquadratic complexity on sets whose similarity rankings arise from a ``generic'' linear order on the $\binom{n}{2}$ inter-point distances in a metric space.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 1 Pith paper
-
FAMST: Fast Approximate Minimum Spanning Tree Construction for Large-Scale and High-Dimensional Data
A three-phase approximate MST algorithm using ANN graphs, random component linking, and local edge refinement achieves near-linear scaling with small error on large high-dimensional data.
Reference graph
Works this paper leans on
-
[7]
Experiments with comparator-based nearest neighbor descent
R W R Darling. Experiments with comparator-based nearest neighbor descent. In preparation, 2020
work page 2020
-
[1]
An extreme value theory for sequence matching
Richard Arratia; Louis Gordon; Michael Waterman. An extreme value theory for sequence matching. Annals of Statistics, Volume 14, Number 3 971-993, 1986
work page 1986
-
[2]
S. Arya; D. M. Mount; N. S. Netanyahu; R. Silverman; A. Y. Wu. An optimal algorithm for approximate nearest neighbor searching fixed dimensions. Journal of the ACM, Volume 45, pages 891–923, 1998
work page 1998
-
[3]
D Bakkelund. An LCS-based string metric. Unpublished notes, University of Oslo. 2009
work page 2009
-
[4]
Etienne Becht; Leland McInnes; John Healy; Charles-Antoine Dutertre; Immanuel W. H. Kwok; Lai Guan Ng; Florent Ginhoux; Evan W. Newell. Dimensionality reduction for visualizing single-cell data using UMAP. Nature Biotechnology volume 37, pages 38—44, 2019
work page 2019
-
[5]
Benchmarks of approximate nearest neighbor libraries in Python
Erik Bernhardsson. Benchmarks of approximate nearest neighbor libraries in Python. github.com/erikbern/ann-benchmarks
-
[6]
B. Bollob´ as; W. F. de la Vega. The diameter of random regular graphs. Combinatorica 2, 125-134, 1982
work page 1982
-
[8]
prank2xy: Clustering and low dimensional projection of ranking systems
R W R Darling. prank2xy: Clustering and low dimensional projection of ranking systems. github.com/probabilist-us/prank2xy, 2020
work page 2020
Show all 29 references
-
[9]
Mirrokni
Mayur Datar; Nicole Immorlica; Piotr Indyk; Vahab S. Mirrokni. Locality-sensitive hashing scheme based on p-stable distributions. In Proceedings of the twentieth annual symposium on computational geometry (SCG ’04). ACM, New York, NY, USA, 253-262, 2004
2004
-
[10]
A Probabilistic Theory of Pattern Recognition
Luc Devroye; Laszlo Gy¨ orfi; Gabor Lugosi. A Probabilistic Theory of Pattern Recognition . Springer Science & Business Media, 2013
2013
-
[11]
Efficient k-nearest neighbor graph construction for generic simi- larity measures
Dong, Wei; Charikar, Moses; Li, Kai. Efficient k-nearest neighbor graph construction for generic simi- larity measures. Proceedings of the 20th International Conference on World Wide Web, 577–586, 2011
2011
-
[12]
Ester; H.-P
M. Ester; H.-P. Kriegel; J. Sander; X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. Proc. 2nd Int. Conf. Knowl. Discov. Data Mining, 226—231, 1996
1996
-
[13]
The diameter of sparse random graphs
Daniel Fernholz; Vijaya Ramachandran. The diameter of sparse random graphs. Random Structures and Algorithms 31, 482-516, 2007
2007
-
[14]
Goyal; Y
N. Goyal; Y. Lifshits; H. Schutze. Disorder inequality: a combinatorial approach to nearest neighbor search. WSDM ’08 Proc. Intern. Conf. Web Search Web Data Mining 25—32, 2008
2008
-
[15]
Comparison based nearest neighbor search
Siavash Haghiri; Debarghya Ghoshdastidar; Ulrike von Luxburg. Comparison based nearest neighbor search. arXiV: 1704.01460, 2017
2017 arXiv
-
[16]
Houle; Michael Nett
Michael E. Houle; Michael Nett. Rank-based similarity search: reducing the dimensional dependence. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 136-150, 2015
2015
-
[17]
Fast perfect sampling from linear extensions
Mark Huber. Fast perfect sampling from linear extensions. Discrete Mathematics 306. 420 – 428, 2006
2006
-
[18]
Indyk; R
P. Indyk; R. Motwani. Approximate nearest neighbors: towards removing the curse of dimensionality. Thirteenth Symposium on Theory of Computing , pages 604-–613, 1998
1998
-
[19]
Foundations of Modern Probability , 2nd ed
Olav Kallenberg. Foundations of Modern Probability , 2nd ed. Springer, New York 2001
2001
-
[20]
Kleindessner; U
M. Kleindessner; U. von Luxburg. Uniqueness of ordinal embedding. JMLR: Workshop and Conference Proceedings vol 35:1–28, 2014
2014
-
[21]
Kleindessner; U
M. Kleindessner; U. von Luxburg. Lens depth function and k-relative neighborhood graph: versatile tools for ordinal data analysis. Journal of Machine Learning Research 18, 1-52, 2017
2017
-
[22]
Krivelevich
M. Krivelevich. Expanders: How to find them, and what to find in them. Surveys in Combinatorics, ed. A. Lo, R. Mycroft, G. Perarnau, A. Treglown, Cambridge University Press 2019
2019
-
[23]
pynndescent: A Python nearest neighbor descent for approximate nearest neigh- bors
Leland McInnes. pynndescent: A Python nearest neighbor descent for approximate nearest neigh- bors. github.com/lmcinnes/pynndescent, 2018
2018
-
[24]
UMAP: uniform manifold approximation and projection for dimension reduction
Leland McInnes; John Healy; James Melville. UMAP: uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426, 2018
2018 arXiv
-
[25]
Marius Muja; David G. Lowe. Scalable nearest neighbor algorithms for high dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36, 2227–2240, 2014
2014
-
[26]
McKay; R
B. McKay; R. W. Robinson. Asymptotic enumeration of Eulerian circuits in the complete graph. Combinatorica: 10, no. 4, pages 367—377, 1995 30 K-NEAREST NEIGHBOR APPROXIMATION VIA THE FRIEND-OF-A-FRIEND PRINCIPLE
1995
-
[27]
Tschopp; S
D. Tschopp; S. Diggavi; P. Delgosha. Randomized Algorithms for Comparison-based Search. NIPS 2011
2011
-
[28]
Salil P. Vadhan. Pseudorandomness. Foundations and Trends in Theoretical Computer Science: Vol. 7,No. 1–3, pp 1–336, 2012
2012
-
[29]
J. H. van Lint; R. M. Wilson. A Course in Combinatorics (2nd ed.). Cambridge University Press, 2001
2001
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.