REVIEW 4 major objections 4 minor 30 references
Large-Scale Graph Building in Dynamic Environments: Low Latency and High Quality
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Dynamic GUS builds and updates a learned multimodal graph in tens of milliseconds, reproducing Grale's neighborhoods exactly when all shared-bucket candidates are retrieved and matching or improving on Grale's edge quality with a finite…
desk verdict The paper's core idea—encode Grale's LSH buckets as sparse features and retrieve candidates with ScaNN—is clean and plausibly useful, but the evaluation never tests the thing that matters: whether the truncated neighborhoods preserve the edges downstream tasks actually need. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the sparse embedding $M(p)$ whose non-zero dimensions are the bucket IDs assigned to $p$ by the locality-sensitive bucketing that Grale would use, with weights set either to $1.0$ or to the logarithm of the inverse document frequency of the bucket. Distance between two points is minus the number of shared weighted dimensions, so two points are close in the embedding exactly when they share a bucket, which is precisely the condition for Grale to treat them as a scoring pair. This identity is what lets a dynamic approximate-nearest-neighbor index such as ScaNN act as the candidate generator, with the trained similarity model applied only to the retrieved candidates. Filtering of overly popular buckets and IDF reweighting preserve the identity (Lemma 4.1 holds for any strictly positive weights) while pruning low-value candidates and reducing memory.
What would settle it
A direct test would be to take a point, retrieve its top candidate set at a given ScaNN-NN budget, then score all other points that share a bucket with it but were not retrieved, and check how many of the model's highest-weight edges per point are missing; if a substantial fraction of the top-100 edges comes from unretrieved candidates as the budget shrinks, the quality claim fails. A companion test would measure downstream task accuracy, such as node classification on ogbn-arxiv, of the dynamic graph versus the from-scratch Grale graph.
Extended reading notes
Core claim
The paper's core discovery is that Grale's learned graph neighborhoods can be reproduced in a dynamic setting without recomputing scoring pairs. Lemma 4.1 states that for any point $p$, the neighborhood produced by Grale equals the neighborhood produced by Dynamic GUS when Dynamic GUS retrieves all points with negative distance in the sparse embedding $M$, where the distance between two points is the negative dot product of their embeddings and each non-zero dimension is a bucket ID. Because two points share a bucket exactly when they share a non-zero dimension, negative distance coincides exactly with being a Grale scoring pair. The system then uses ScaNN to retrieve a bounded number of nearest neighbors, scores those candidates with the trained similarity model, and returns the top edges. The empirical claim is that, on ogbn-products and ogbn-arxiv, the resulting edge-weight distributions are comparable to or better than Grale's, with median query latencies of 5 to 25 ms and insertion latencies under a millisecond.
Load-bearing premise
The system's quality claim rests on the assumption that the fixed number of nearest neighbors returned by ScaNN contains all pairs the trained model would rank as important neighbors, but Lemma 4.1 guarantees equivalence only when every point with negative distance is retrieved, and no bound is proved on the loss from truncation.
Editorial extensions
If this is right
- With all negative-distance candidates retrieved, the graph built dynamically is provably identical to the graph Grale would build from scratch, as stated by Lemma 4.1.
- With a bounded candidate budget, edge-weight distributions are comparable to or better than Grale's on ogbn-products and slightly lower on ogbn-arxiv, while queries take 5 to 25 ms median latency.
- Insertions and updates take under a millisecond on the tested datasets, so newly arriving points can affect neighborhoods within seconds.
- The design is agnostic to the specific similarity model: any trained pairwise scorer can be applied to the retrieved candidates.
- The system supports hundreds of thousands of mutations and neighborhood queries per second, making dynamic graphs feasible for applications such as app-risk detection, where the paper reports harmful apps are captured four times faster.
Reading between the lines
- Because Lemma 4.1 holds for any embedding with strictly positive bucket weights, the argument is not tied to ScaNN: any dynamic sparse-vector ANN index could be substituted, with quality then governed by that index's candidate recall.
- The paper leaves open a formal worst-case bound on the quality loss from truncating the candidate set; a natural extension is a radius- or recall-based guarantee relating the retrieval budget to the fraction of high-weight edges recovered.
- The reported four-fold speedup in capturing harmful apps likely reflects data freshness rather than raw graph quality, which suggests that downstream task metrics, not edge-weight distributions, are the decisive test for such applications.
- A testable extension is to report downstream task accuracy for GUS-built graphs versus Grale-built graphs on the same benchmarks; the paper does not include such metrics.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Dynamic GUS, a system for building and maintaining large graphs in dynamic environments. The approach converts multimodal point features into sparse embeddings whose nonzero coordinates correspond to Grale-style LSH bucket IDs, uses ScaNN to retrieve approximate nearest neighbors under negative dot-product distance, and then scores the retrieved candidates with a trained similarity model. The central claim is that Dynamic GUS inherits Grale's graph quality while supporting low-latency insertions, deletions, and neighborhood queries. The paper proves a formal equivalence (Lemma 4.1) between Grale's scoring pairs and Dynamic GUS's candidate set when all negative-distance points are retrieved, validates this equivalence experimentally on ogbn-products and ogbn-arxiv (Fig. 3), and compares edge-weight distributions with Grale under various truncation and preprocessing configurations (Figs. 4-8). Latency and resource measurements are reported for a dynamic setting (Figs. 9-10).
Significance. If the quality claim holds, Dynamic GUS would be a practically important contribution: it addresses a real gap by combining the multimodal flexibility of Grale with the dynamic, low-latency operation of ANN indexes, and the reported deployment evidence (over 10 applications, including Android security) suggests substantial industrial utility. The paper has clear strengths: Lemma 4.1 is stated precisely and its proof is correct, the offline validation in Fig. 3 is a good methodological step, the latency experiments are detailed and use public datasets, and the system design is cleanly decomposed into embedding generation, candidate retrieval, and similarity scoring. However, the central quality claim depends on the assumption that the truncated ScaNN candidate set contains the edges that Grale would rank highly, and this assumption is not directly tested. The edge-weight percentile plots compare sets of very different sizes and cannot detect missing high-value edges; no downstream task metrics are reported. The stress-test concern that the truncation loss is unquantified is valid and is the main obstacle to accepting the paper's headline claim.
major comments (4)
- [4.1] Lemma 4.1 establishes exact equivalence between Grale's scoring pairs and Dynamic GUS's candidate set only when all points with negative distance are retrieved. In the deployed and evaluated system, however, candidate retrieval is bounded to ScaNN-NN in {10, 100, 1000} (Sections 4.1 and 5.1). The manuscript provides no theorem, no measured recall bound, and no analysis quantifying the loss incurred by this truncation relative to the full negative-distance set. Consequently, the abstract's claim that Dynamic GUS 'inherits the advantages and the quality of Grale' is not supported for the configurations actually used in the evaluation.
- [5.1, Figs. 4 and 5] The edge-weight percentile comparisons are not a valid substitute for measuring whether Dynamic GUS recovers the edges that Grale would score highly. The total number of edges differs by orders of magnitude across configurations; for example, Fig. 4(e) reports 2,446,579,971 edges for ogbn-products with Filter-P=0 versus 135,807,863 edges with Filter-P=10, and Fig. 6(a) reports 140,242,620,100 edges for Grale versus 19,020,281 for GUS. An algorithm that returns only its top-scoring edges will appear above Grale on these plots even if it drops most structurally important neighbors. The paper itself instructs readers to take the total edge count into account, but the subsequent quality claims are based on the raw percentile plots without a recall or precision measure.
- [5.1, Third Experiment (Figs. 5 and 8)] The Top-K post-processing experiment does not resolve the truncation concern. For ogbn-arxiv the authors state that the scores of edges found by Dynamic GUS are 'slightly lower' than Grale's, and the total edge counts still differ substantially in the reported configurations (e.g., Fig. 5(a): 20,866,944 for GUS vs. 24,490,290 for Grale on ogbn-products). No downstream task metrics (such as node classification, clustering, or retrieval quality) are reported, so the paper does not establish that the truncated neighborhoods preserve the properties that make Grale useful in downstream graph mining applications.
- [5.1, First Experiment (Fig. 3)] The validation of Lemma 4.1 is performed without Grale's bucket-splitting mechanism and with all negative-distance points retrieved from ScaNN. This is not the regime in which the system operates: the actual configurations use a bounded ScaNN-NN and Grale is run with Bucket-S=1000. The experiment confirms the lemma but does not test the truncated regime, so it cannot by itself support the paper's central quality claim about the deployed system.
minor comments (4)
- [Abstract] The phrase 'Such setting make the use of Grale prohibitive' contains a subject-verb agreement error; it should read 'Such settings make the use of Grale prohibitive.'
- [Fig. 5 caption] The caption reports 'ScaNN – Top-K=10 – NN=100 – IDF-S=0 – Filter-P=10' for the GUS configuration, but Section 5.1 states that ScaNN-NN is set equal to K, so the value should be 10, not 100. This appears to be a typographical inconsistency.
- [Fig. 8 caption] The caption uses 'Top=K=100' instead of 'Top-K=100'.
- [References] Several references have empty date fields before the '[Accessed ...]' strings; this should be cleaned up for the camera-ready version.
Circularity Check
Lemma 4.1 is a definitional equivalence true by construction, but the central latency and bounded-retrieval quality claims are not circular.
-
self definitional
[Section 4.1, Lemma 4.1; echoed in Section 5.1, first experiment]
"For any point p, the neighborhood of p is exactly the same in Grale and Dynamic GUS if we retrieve all the points with negative distance to p in ScaNN. ... Dist(p1, p2) < 0 if and only if the points share a non-zero dimension, which happens if and only if they share a bucket ID. By definition of Grale, two points are considered a scoring pair if and only if they share a bucket ID."
The embedding M is defined from Grale's bucket IDs: 'The embedding of point p has ℓ non-zero dimensions. These dimensions are b1, b2, . . . bℓ, and their weights are 1.0.' Since Dist(p1,p2) = -M(p1)·M(p2), negative distance means exactly that the two embeddings share a non-zero dimension, which means exactly that the points share a bucket ID. Grale's scoring-pair predicate is, by definition, also 'share a bucket ID'. Therefore Lemma 4.1 is not an independent derivation but a restatement of the definitions; the identical curves in Fig. 3 are forced by construction. The finite-retrieval (ScaNN-NN) comparisons do not reduce to this identity, so the step is not load-bearing for the main empirical claim.
full rationale
Score 2 rather than 0 because Lemma 4.1 is a formal result whose conclusion is already contained in the definitions of the embedding and of Grale's scoring pairs. This is a minor definitional equivalence, not a fabricated prediction. The central contributions are independent of it: the paper measures quality under bounded ScaNN-NN retrieval with filtering and IDF, comparing GUS edges against Grale edges using the same trained similarity model, and reports latency and resource usage. No fitted parameter is renamed as a prediction, and no load-bearing self-citation or imported uniqueness theorem appears; the citations to Grale and ScaNN are normal prior work with externally specified algorithms. The skeptic's concern that finite candidate retrieval may omit high-value Grale edges is a correctness and evaluation gap, not a circularity, because that claim is assessed empirically rather than derived from its own assumptions.
Assumptions & free parameters
free parameters (3)
- Filter-P =
0 or 10 (%)
- IDF-S =
0, 1e6, 1e7, or 1e8
- ScaNN-NN =
10, 100, or 1000
assumptions (3)
- domain assumption Points sharing an LSH bucket are likely to be similar under the target similarity notion
- domain assumption The trained two-layer neural network is a valid scoring function for graph quality
- domain assumption ScaNN's ANN search is accurate enough in the dynamic setting
Cite this review
Pith. "Pith review of Large-Scale Graph Building in Dynamic Environments: Low Latency and High Quality." pith.science (2026). https://pith.science/paper/3MCXIODP
@misc{pith2026250710139,
author = {Pith},
title = {Pith review of: Large-Scale Graph Building in Dynamic Environments: Low Latency and High Quality},
year = {2026},
howpublished = {\url{https://pith.science/paper/3MCXIODP}},
note = {Machine review of arXiv:2507.10139}
}
read the original abstract
Learning and constructing large-scale graphs has attracted attention in recent decades, resulting in a rich literature that introduced various systems, tools, and algorithms. Grale is one of such tools that is designed for offline environments and is deployed in more than 50 different industrial settings at Google. Grale is widely applicable because of its ability to efficiently learn and construct a graph on datasets with multiple types of features. However, it is often the case that applications require the underlying data to evolve continuously and rapidly and the updated graph needs to be available with low latency. Such setting make the use of Grale prohibitive. While there are Approximate Nearest Neighbor (ANN) systems that handle dynamic updates with low latency, they are mostly limited to similarities over a single embedding. In this work, we introduce a system that inherits the advantages and the quality of Grale, and maintains a graph construction in a dynamic setting with tens of milliseconds of latency per request. We call the system Dynamic Grale Using ScaNN (Dynamic GUS). Our system has a wide range of applications with over 10 deployments at Google. One of the applications is in Android Security and Privacy, where Dynamic Grale Using ScaNN enables capturing harmful applications 4 times faster, before they can reach users.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
https://github.com/facebookresearch/pysparnn,
GitHub - facebookresearch/pysparnn: Approximate Nearest Neighbor Search for Sparse Data in Python! — github.com. https://github.com/facebookresearch/pysparnn, . [Accessed 25-07-2024]
work page 2024
-
[2]
GitHub - ke-li/dci-knn: Fast k-Nearest Neighbour Search using Dynamic Continuous Indexing (DCI) — github.com.https://github.com/ke-li/dci-knn, . [Accessed 25-07-2024]
work page 2024
-
[3]
https://github.com/pgvector/pgvector,
GitHub - pgvector/pgvector: Open-source vector similarity search for Postgres — github.com. https://github.com/pgvector/pgvector, . [Accessed 25-07-2024]
work page 2024
-
[4]
https://github.com/yahoojapan/NGT,
GitHub - yahoojapan/NGT: Nearest Neighbor Search with Neighborhood Graph and Tree for High-dimensional Data — github.com. https://github.com/yahoojapan/NGT, . [Accessed 25-07-2024]
work page 2024
-
[5]
Additive quantization for extreme vector compression
Artem Babenko and Victor Lempitsky. Additive quantization for extreme vector compression. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 931–938, 2014
work page 2014
- [6]
-
[7]
Learning from labeled and unlabeled data using graph mincuts
Avrim Blum and Shuchi Chawla. Learning from labeled and unlabeled data using graph mincuts. In Carla E. Brodley and Andrea Pohoreckyj Danyluk, editors,Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), Williams College, Williamstown, MA, USA, June 28 - July 1, 2001, pages 19–26. Morgan Kaufmann, 2001
work page 2001
-
[8]
Machine Learning on Graphs: A Model and Comprehensive Taxonomy
Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher R´ e, and Kevin Murphy. Machine learning on graphs: A model and comprehensive taxonomy.CoRR, abs/2005.03675, 2020
work page Pith review arXiv 2005
Show all 30 references
-
[9]
Spann: Highly-efficient billion-scale approximate nearest neighbor search
Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. Spann: Highly-efficient billion-scale approximate nearest neighbor search. In 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021
2021
-
[10]
Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks
Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Ankur Teredesai, Vipin Kumar, Ying Li, R´ omer Rosales, Evimaria Terzi, and George Karypis, editors, P...
2019
-
[11]
Performance of recommender algorithms on top-n recommendation tasks
Paolo Cremonesi, Yehuda Koren, and Roberto Turrin. Performance of recommender algorithms on top-n recommendation tasks. In Xavier Amatriain, Marc Torrens, Paul Resnick, and Markus Zanker, editors,Proceedings of the 2010 ACM Conference on Recommender Systems, RecSys 2010, Barce...
2010
-
[12]
de Sousa, Solange O
Celso Andr´ e R. de Sousa, Solange O. Rezende, and Gustavo E. A. P. A. Batista. Influence of graph construction on semi-supervised learning. In Hendrik Blockeel, Kristian Kersting, Siegfried Nijssen, and Filip Zelezn´ y, editors,Machine Learning and Knowledge Discovery in Data...
2013
-
[13]
Dean, Mark A
Thomas L. Dean, Mark A. Ruzon, Mark Segal, Jonathon Shlens, Sudheendra Vijayanarasimhan, and Jay Yagnik. Fast, accurate detection of 100, 000 object classes on a single machine. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, June 23-28, ...
2013
-
[14]
Optimized product quantization for approximate nearest neighbor search
Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. Optimized product quantization for approximate nearest neighbor search. In2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, June 23-28, 2013, pages 2946–2953. IEEE Computer Society, 2013
2013
-
[15]
Accelerating large-scale inference with anisotropic vector quantization
Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. Accelerating large-scale inference with anisotropic vector quantization. InProceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Ev...
2020
-
[16]
Grale: Designing networks for graph learning
Jonathan Halcrow, Alexandru Mosoi, Sam Ruth, and Bryan Perozzi. Grale: Designing networks for graph learning. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, ...
2020
-
[17]
Billion-scale similarity search with gpus.IEEE Trans
Jeff Johnson, Matthijs Douze, and Herv´ e J´ egou. Billion-scale similarity search with gpus.IEEE Trans. Big Data, 7(3):535–547, 2021
2021
-
[18]
Fast k-nearest neighbour search via Dynamic Continuous Indexing
Ke Li and Jitendra Malik. Fast k-nearest neighbour search via Dynamic Continuous Indexing. InInternational Conference on Machine Learning, pages 671–679, 2016
2016
-
[19]
Malkov and D
Yu A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):824–836, 2020. doi: 10.1109/TPAMI.2018.2889473
2020
-
[20]
On potts model clustering, kernel k-means and density estimation.Journal of Computational and Graphical Statistics, 17(3): 629–658, 2008
Alejandro Murua, Larissa Stanberry, and Werner Stuetzle. On potts model clustering, kernel k-means and density estimation.Journal of Computational and Graphical Statistics, 17(3): 629–658, 2008
2008
-
[21]
Learning and inference via maximum inner product search
Stephen Mussmann and Stefano Ermon. Learning and inference via maximum inner product search. In Maria-Florina Balcan and Kilian Q. Weinberger, editors,Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, vol...
2016
-
[22]
Large scale distributed semi-supervised learning using streaming approximation
Sujith Ravi and Qiming Diao. Large scale distributed semi-supervised learning using streaming approximation. InArtificial intelligence and statistics, pages 519–528. PMLR, 2016
2016
-
[23]
Soar: improved indexing for approximate nearest neighbor search.Advances in Neural Information Processing Systems, 36, 2024
Philip Sun, David Simcha, Dave Dopson, Ruiqi Guo, and Sanjiv Kumar. Soar: improved indexing for approximate nearest neighbor search.Advances in Neural Information Processing Systems, 36, 2024
2024
-
[24]
Scalable k-nn graph construction for visual descriptors
Jing Wang, Jingdong Wang, Gang Zeng, Zhuowen Tu, Rui Gan, and Shipeng Li. Scalable k-nn graph construction for visual descriptors. InCVPR 2012, pages 1106–1113, 2012
2012
-
[25]
Query-driven iterated neighborhood graph search for large scale indexing
Jingdong Wang and Shipeng Li. Query-driven iterated neighborhood graph search for large scale indexing. InACM Multimedia 2012, pages 179–188, 2012
2012
-
[26]
Microsoft academic graph: When experts are not enough.Quant
Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Microsoft academic graph: When experts are not enough.Quant. Sci. Stud., 1(1): 396–413, 2020
2020
-
[27]
A quest for structure: Jointly learning the graph structure and semi-supervised classification
Xuan Wu, Lingxiao Zhao, and Leman Akoglu. A quest for structure: Jointly learning the graph structure and semi-supervised classification. In Alfredo Cuzzocrea, James Allan, Norman W. Paton, Divesh Srivastava, Rakesh Agrawal, Andrei Z. Broder, Mohammed J. Zaki, K. Sel¸ cuk Cand...
2018
-
[28]
Cohen, and Ruslan Salakhutdinov
Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In Maria-Florina Balcan and Kilian Q. Weinberger, editors,Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA...
2016
-
[29]
Yu, Daniel Niels Holtmann-Rice, Sanjiv Kumar, and Pradeep Ravikumar
Ian En-Hsu Yen, Satyen Kale, Felix X. Yu, Daniel Niels Holtmann-Rice, Sanjiv Kumar, and Pradeep Ravikumar. Loss decomposition for fast learning in large output spaces. In Jennifer G. 15 Dy and Andreas Krause, editors,Proceedings of the 35th International Conference on Machine ...
2018
-
[30]
Learning with local and global consistency
Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Sch¨ olkopf. Learning with local and global consistency. In Sebastian Thrun, Lawrence K. Saul, and Bernhard Sch¨ olkopf, editors,Advances in Neural Information Processing Systems 16 [Neural Informati...
2003
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.