REVIEW 3 major objections 4 minor 96 references
The paper establishes that numerically exact metric magnitude weightings can be computed at scales approaching 10^5 points by solving the dense system Zw=1 with hierarchical low-rank solvers, while sparsification and Nyström approximations
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-07-31 23:38 UTC pith:LL2XQA63
load-bearing objection Useful benchmark, but the central claim about numerically exact HSS solves at 10^5 scale is unsupported without an accuracy check against dense solutions. the 3 major comments →
Scalably computing metric magnitude
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the weighting equation Zw=1, with Z_jk = exp(−t∥x_j−x_k∥), can be solved exactly and scalably using hierarchical low-rank (HSS) matrix solvers. For n up to about 30,000, a single iteration of the cutoff root-finding takes about a minute; the practical ceiling for the current build is n≈65,000, with a clear path to 10^5 on a powerful workstation. Sparsification by nearest-neighbor thresholding produces low correlations with exact results and destroys the positive definiteness needed for information-theoretic guarantees. Nyström approximations yield weightings that are only accidentally accurate at one unknown scale. The authors conclude that structured kernel solvers
What carries the argument
The load-bearing object is the similarity matrix Z=exp(−td) built from a Euclidean distance matrix d, whose positive definiteness guarantees a unique weighting w solving Zw=1. Because the diversity-maximizing distribution is proportional to w, exactness matters. The proposed mechanism is a hierarchical low-rank (HSS) compression and factorization of Z, which exploits kernel smoothness to reduce cost from cubic to subquadratic, combined with root-finding for the cutoff t+ where w first becomes nonnegative.
Load-bearing premise
The claim that 10^5 points are reachable rests on the assumption that the segfault near n=70,000 is a bug in the BLAS library rather than a sign that HSS compression ranks or memory use grow uncontrollably for this kernel.
What would settle it
Run the identical pipeline at n=70,000 with a different BLAS implementation or with varying compression tolerances. If the segfault persists or the HSS rank at the root of the tree grows superlinearly, the path to n=10^5 collapses. Conversely, successfully solving n=100,000 within memory on a 128 GB workstation would confirm the claim.
If this is right
- Exact weightings become computable at n≈10^5 on a single workstation, and larger with MPI/CUDA clusters.
- Diversity-maximizing distributions, which require only w, can be produced for large point sets across many ML tasks.
- The documented failure modes of sparsification and Nyström redirect future scaling efforts toward structured solvers.
- The approach also handles the t=0 limit via distance matrix solves, broadening applicability.
- A publicly released containerized pipeline enables reproducible reuse of the method.
Where Pith is reading between the lines
- If HSS compression ranks grow faster than expected for Gaussian kernels in higher dimension, the 10^5 ceiling may be optimistic; a test at n=100,000 in m=100 would settle this.
- The segfault near n=70,000 is likely a fixable infrastructure bug, but it may signal memory pressure from large HSS ranks; tracking rank growth as n increases would separate the two.
- For downstream tasks like black-box optimization, approximate weightings might suffice even if exact solves do not scale, so the paper's negative results on approximation should not be read as blocking all applications.
- The empirical scaling law for cutoff on Gaussian data (t+ ≈ γ + m^(−2/3) n^(1/4)) could seed root-finding initial guesses, though the paper finds little time savings; a more careful implementation might extract more.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses the scalability of computing metric magnitude weightings w satisfying Z w = 1 for Euclidean point clouds, where Z = exp(-t d). It benchmarks three approaches: sparsification, Nyström approximation, and hierarchical low-rank (HSS) solvers via STRUMPACK, using Gaussian datasets with n up to 30,000. The authors report that sparsification and Nyström fail, while HSS solvers offer better runtime and memory scaling than dense solvers, and they claim a clear path to n ≈ 10^5 on a single workstation and larger scales with MPI/CUDA. They also present an empirical law E t_+ ≈ γ_+ m^{-2/3} n^{1/4} for Gaussian cutoff scales and document a segfault at n ≈ 70,000 attributed to OpenBLAS.
Significance. If correct, the HSS-based pipeline would significantly widen the applicability of numerically exact metric magnitude, which is currently limited by dense linear algebra costs. The paper's strengths are its systematic benchmarking across solver families, the release of a Dockerized C++/MPI pipeline, and the documentation of negative results for sparsification and Nyström that may guide future work. However, the central numerical claim is currently supported only by runtime and memory measurements; the accuracy of the HSS weightings is never validated against dense solutions, which is essential for the cutoff t_+ and for the 'numerically exact' framing.
major comments (3)
- [§4.3 and §2] No accuracy validation of HSS weightings is reported. Figures 4–7 show runtime, memory, and CPU utilization only; there is no comparison of HSS-computed w or t_+ against dense solutions for n where dense solves are feasible. This matters because the root-finder in §2 halts when min_j w_j ∈ (0, ε), and t_+ is defined by the sign of weighting components. Small HSS approximation errors can flip signs or shift the root. The statement in §5.1 that 'loose HSS tolerances led to unacceptable noise' is not quantitative. The authors should report the HSS tolerance used and error metrics (e.g., max |w_HSS - w_dense|, sign agreement, t_+ difference) for n = 1000, 3000, and if possible 10000.
- [§5.1 and Abstract] The abstract claims 'a clear path to scales of 10^5 points on a single powerful workstation,' but the largest successful experiment is n = 30,000, and the next data point is a segfault at n ≈ 70,000. The paper attributes the crash to OpenBLAS, yet immediately adds that 'the underlying cause is likely large HSS ranks and/or compression tolerances.' These two explanations are in tension. If HSS ranks/compression are the limiting factor, the BLAS attribution is not load-bearing and the path to 10^5 is unsupported. The authors should either provide successful runs at n ≈ 50,000–100,000, or give a rank/tolerance-based scaling analysis that justifies the extrapolation.
- [§4.2 and Abstract] The reproducibility claim is internally inconsistent. The abstract and §1 state that the pipeline is 'open-source' at https://github.com/Cynnovative/compass, while §4.2 says 'All code, build recipes, environment files, and Docker images ... are in GitLab and will be publicly released.' If the code is not currently available, the reproducibility claim is premature; if it is available, the location should be unambiguous. This is a presentation issue but undermines a stated contribution.
minor comments (4)
- [§C.2, Eq. (8)] Equation (8) is presented as an 'excellent approximation,' but γ_+ is fit to the same data shown in Figures 13–14. This is a fitted curve, not an independent prediction. The paper candidly says it is not clear how to exploit it, so this does not affect the main scaling conclusion, but the wording 'excellent approximation' should be tempered and the fitted nature stated.
- [§4.3] The sentence on iterative methods ('iterative methods ... solved as fast or faster than the dense pipeline' but 'only marginally beneficial') is not backed by any displayed experiment. Either add a small table or remove the claim.
- [§3.1 and Appendix A] The sparsification failure is reported with a single n=1024, m=32 example (Figure 8) and a statement about Kendall tau 'overwhelmingly below 0.85.' Additional details on how many configurations were tested and what 'overwhelmingly' means would strengthen the negative result.
- [Throughout] The paper uses 'numerically exact' loosely while relying on a tolerance-controlled HSS solver. Consider replacing with 'direct solves' or 'high-accuracy solves' unless accuracy is demonstrated.
Circularity Check
No significant circularity: central scaling comparison is benchmarked against an external dense solver and no load-bearing prediction reduces to its inputs.
full rationale
The paper's central claim—that hierarchical low-rank (HSS/STRUMPACK) solvers are preferable for computing metric magnitude weightings at scale—is supported by runtime, memory, and CPU comparisons against SciPy's dense solver on the same data (Figures 4-7). This is an external benchmark, not a self-referential construction. The root-finding procedure for t+ uses the definitional property w>=0 and an elementary bound stated in the text; the bound's self-citation (Huntsman, 2023a) is not load-bearing because the inequality is simple and independently presented. Sparsification and Nyström failures are evaluated by comparison to exact dense solutions, so they are not circular. The empirical law (8), Et+ ≈ gamma+ m^{-2/3} n^{1/4}, is explicitly presented as a numerical observation ('a numerical experiment suggests'), with gamma+ selected to match the same data (Figure 13, 'taking gamma+ ≈ 2^{7/8}'); it is a fitted curve rather than a derived prediction, and the paper itself states that exploiting it appears to save no time and that its utility is unclear. Thus it is not a load-bearing prediction. The acknowledged lack of explicit accuracy validation for HSS solutions (the skeptic's concern) is a correctness or robustness risk, not a circularity. Overall, the derivation chain is self-contained against an external dense-solver baseline, so no circular step is established.
Axiom & Free-Parameter Ledger
free parameters (3)
- gamma_plus =
≈2^{7/8} ≈ 1.834
- scaling_exponents =
−2/3 (in m), 1/4 (in n)
- root_finder_stop_tolerance =
ε comparable to machine epsilon
axioms (5)
- standard math Z = exp(-t d) is positive definite for Euclidean distance matrices, guaranteeing a unique weighting.
- domain assumption STRUMPACK HSS solves return solutions close enough to exact solutions to locate t+ reliably at the tolerances used.
- ad hoc to paper The OpenBLAS dgemm segfault at n≈70,000 is a BLAS bug, not an HSS/STRUMPACK limitation.
- domain assumption Gaussian-distributed test data is representative of the Euclidean point clouds that metric-magnitude applications target.
- ad hoc to paper Eq (8) provides a valid empirical approximation E t+ ≈ γ+ m^{-2/3} n^{1/4} for Gaussian cutoffs.
read the original abstract
Applications of metric magnitude often rely on numerically exact results in order to exploit a connection with information theory. We examine various approaches for scaling the dense linear algebra involved and identify hierarchical low-rank solvers as a preferred approach, with a clear path to scales of $10^5$ points on a single powerful workstation, and larger scales using our containerized CUDA-enabled C++/MPI pipeline.
Figures
Reference graph
Works this paper leans on
-
[1]
Abedsoltan, Amirhesam and Pandit, Parthe and Rademacher, Luis and Belkin, Mikhail , booktitle=. On the. 2024 , organization=
2024
-
[2]
Journal of Applied and Computational Topology , volume=
The magnitude vector of images , author=. Journal of Applied and Computational Topology , volume=. 2024 , publisher=
2024
-
[3]
Distribution of the distance between two random points in a body from
Aharonyan, NG and Khalatyan, V , journal=. Distribution of the distance between two random points in a body from. 2020 , publisher=
2020
-
[4]
2011 , publisher=
The Oxford Handbook of Random Matrix Theory , author=. 2011 , publisher=
2011
-
[5]
The Annals of Statistics , volume=
Concentration of kernel matrices with application to kernel spectral clustering , author=. The Annals of Statistics , volume=. 2021 , publisher=
2021
-
[6]
Symposium on Discrete Algorithms , year=
Overcoming the _1 non-embeddability barrier: algorithms for product metrics , author=. Symposium on Discrete Algorithms , year=
-
[7]
Topological, Algebraic and Geometric Learning Workshops 2023 , year=
Metric space magnitude and generalisation in neural networks , author=. Topological, Algebraic and Geometric Learning Workshops 2023 , year=
2023
-
[8]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Approximating metric magnitude of point sets , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[9]
Journal of Machine Learning Research , volume=
On the behavior of intrinsically high-dimensional spaces: distances, direct and reverse nearest neighbors, and hubness , author=. Journal of Machine Learning Research , volume=
-
[10]
2010 , publisher=
Spectral Analysis of Large Dimensional Random Matrices , author=. 2010 , publisher=
2010
-
[11]
Journal of Computational Biophysics and Chemistry , volume=
Persistent magnitude for the quantitative analysis of the structure and stability of carboranes , author=. Journal of Computational Biophysics and Chemistry , volume=. 2024 , publisher=
2024
-
[12]
Royal Society Open Science , volume=
Topological magnitude for protein flexibility analysis , author=. Royal Society Open Science , volume=. 2025 , publisher=
2025
-
[13]
arXiv preprint arXiv:0710.2063 , year=
Distance matrices and isometric embeddings , author=. arXiv preprint arXiv:0710.2063 , year=
Pith/arXiv arXiv 2063
-
[14]
Bordenave, Charles , journal=. On
-
[15]
ACM Computing Surveys (CSUR) , volume=
Outlier detection: methods, models, and classification , author=. ACM Computing Surveys (CSUR) , volume=. 2020 , publisher=
2020
-
[16]
arXiv preprint arXiv:2006.14063 , year=
Practical applications of metric space magnitude and weighting vectors , author=. arXiv preprint arXiv:2006.14063 , year=
Pith/arXiv arXiv 2006
-
[17]
arXiv preprint arXiv:2106.00827 , year=
Weighting vectors for machine learning: numerical harmonic analysis applied to boundary detection , author=. arXiv preprint arXiv:2106.00827 , year=
-
[18]
SIAM Journal on Matrix Analysis and Applications , volume=
Fast deterministic approximation of symmetric indefinite kernel matrices with high dimensional datasets , author=. SIAM Journal on Matrix Analysis and Applications , volume=. 2022 , publisher=
2022
-
[19]
, author=
Characterization, stability and convergence of hierarchical clustering methods. , author=. Journal of Machine Learning Research , volume=. 2010 , url=
2010
-
[20]
Foundations of Computational Mathematics , volume=
Classifying clustering schemes , author=. Foundations of Computational Mathematics , volume=. 2013 , publisher=
2013
-
[21]
ACM Computing Surveys (CSUR) , volume=
Anomaly detection: a survey , author=. ACM Computing Surveys (CSUR) , volume=. 2009 , publisher=
2009
-
[22]
IEEE International Parallel and Distributed Processing Symposium (IPDPS) , pages=
Scalable and memory-efficient kernel ridge regression , author=. IEEE International Parallel and Distributed Processing Symposium (IPDPS) , pages=. 2020 , organization=
2020
-
[23]
arXiv preprint arXiv:2407.15724 , year=
Beyond size and class balance: alpha as a new dataset quality metric for deep learning , author=. arXiv preprint arXiv:2407.15724 , year=
-
[24]
2022 , school=
Graph Geometry from Effective Resistances , author=. 2022 , school=
2022
-
[25]
2009 , publisher=
Encyclopedia of Distances , author=. 2009 , publisher=
2009
-
[26]
Random Matrices: Theory and Applications , volume=
The spectrum of random kernel matrices: universality results for rough and varying kernels , author=. Random Matrices: Theory and Applications , volume=. 2013 , publisher=
2013
-
[27]
The magnitude of dominated sets: a
Emmerich, Michael , journal=. The magnitude of dominated sets: a
-
[28]
1994 , publisher =
Dumb and Dumber , author =. 1994 , publisher =
1994
-
[29]
arXiv preprint arXiv:1504.00253 , year=
Tables of the existence of equiangular tight frames , author=. arXiv preprint arXiv:1504.00253 , year=
-
[30]
Randomized
Frangella, Zachary and Tropp, Joel A and Udell, Madeleine , journal=. Randomized. 2023 , publisher=
2023
-
[31]
Linear Algebra and its Applications , volume=
A constrained eigenvalue problem , author=. Linear Algebra and its Applications , volume=. 1989 , publisher=
1989
-
[32]
Transactions of the American Mathematical Society , volume=
Functions preserving positive definiteness for sparse matrices , author=. Transactions of the American Mathematical Society , volume=
-
[33]
Transactions of the American Mathematical Society , volume=
Preserving positivity for matrices with sparsity constraints , author=. Transactions of the American Mathematical Society , volume=
-
[34]
arXiv preprint arXiv:2101.02055 , year=
Geometric entropic exploration , author=. arXiv preprint arXiv:2101.02055 , year=
-
[35]
2015 , publisher=
Hierarchical Matrices: Algorithms and Analysis , author=. 2015 , publisher=
2015
-
[36]
Linear Algebra and its Applications , volume=
Finite metric spaces of strictly negative type , author=. Linear Algebra and its Applications , volume=. 1998 , publisher=
1998
-
[37]
Benchmarking an algorithm for expensive high-dimensional objectives on the
Hoffman, Zachary and Huntsman, Steve , booktitle=. Benchmarking an algorithm for expensive high-dimensional objectives on the
-
[38]
2012 , publisher=
Matrix Analysis , author=. 2012 , publisher=
2012
-
[39]
arXiv preprint arXiv:2201.11677 , year=
Parallel black-box optimization of expensive high-dimensional multimodal functions via magnitude , author=. arXiv preprint arXiv:2201.11677 , year=
-
[40]
International Conference on Evolutionary Multi-Criterion Optimization , year=
Diversity enhancement via magnitude , author=. International Conference on Evolutionary Multi-Criterion Optimization , year=
-
[41]
arXiv preprint arXiv:2304.08334 , year=
Magnitude of arithmetic scalar and matrix categories , author=. arXiv preprint arXiv:2304.08334 , year=
-
[42]
Genetic and Evolutionary Computation Conference , year=
Quality-diversity in dissimilarity spaces , author=. Genetic and Evolutionary Computation Conference , year=
-
[43]
Proceedings of the 1st Conference on Topology, Algebra, and Geometry in Data Science , year =
Peeling metric spaces of strict negative type , author =. Proceedings of the 1st Conference on Topology, Algebra, and Geometry in Data Science , year =
-
[44]
arXiv preprint arXiv:2603.26645 , year=
Peel neighborhoods , author=. arXiv preprint arXiv:2603.26645 , year=
-
[45]
Environmental and Ecological Statistics , volume=
Quadratic diversity: its maximization can reduce the richness of species , author=. Environmental and Ecological Statistics , volume=. 2002 , publisher=
2002
-
[46]
Hokkaido Mathematical Journal , volume=
Maximization of quadratic forms expressed by distance matrices , author=. Hokkaido Mathematical Journal , volume=. 2006 , publisher=
2006
-
[47]
The Annals of Statistics , pages=
THE SPECTRUM OF KERNEL RANDOM MATRICES , author=. The Annals of Statistics , pages=. 2010 , publisher=
2010
-
[48]
arXiv preprint arXiv:2506.11831 , year=
Bayesian Optimization with Inexact Acquisition: Is Random Grid Search Sufficient? , author=. arXiv preprint arXiv:2506.11831 , year=
-
[49]
arXiv preprint arXiv:2508.01834 , year=
Efficient optimization of expensive black-box simulators via marginal means, with application to neutrino detector design , author=. arXiv preprint arXiv:2508.01834 , year=
-
[50]
SIAM Journal on Computing , volume=
Improved lower bounds for embeddings into L_1 , author=. SIAM Journal on Computing , volume=. 2009 , publisher=
2009
-
[51]
MILCOM , year=
Use of Markov chains for spectral shaping , author=. MILCOM , year=
-
[52]
Ecology , volume=
Measuring diversity: the importance of species similarity , author=. Ecology , volume=. 2012 , url=
2012
-
[53]
Documenta Mathematica , volume=
The magnitude of metric spaces , author=. Documenta Mathematica , volume=
-
[54]
Entropy , volume=
Maximizing diversity in biology and beyond , author=. Entropy , volume=. 2016 , publisher=
2016
-
[55]
2021 , publisher=
Entropy and Diversity , author=. 2021 , publisher=
2021
-
[56]
Journal of Inequalities and Applications , volume=
On the distribution-tail behaviour of the product of normal random variables , author=. Journal of Inequalities and Applications , volume=. 2023 , publisher=
2023
-
[57]
Neural Information Processing Systems , year=
Metric space magnitude for evaluating the diversity of latent representations , author=. Neural Information Processing Systems , year=
-
[58]
Advances in Neural Information Processing Systems , year=
Geometry-aware edge pooling for graph neural networks , author=. Advances in Neural Information Processing Systems , year=
-
[59]
2024 IEEE International Conference on Big Data (BigData) , pages=
Exploring the Semantic Representations of Text in Subspaces of Latent Space: A Case Study on Color , author=. 2024 IEEE International Conference on Big Data (BigData) , pages=. 2024 , organization=
2024
-
[60]
Advances in Neural Information Processing Systems , year=
Core-sets for fair and diverse data summarization , author=. Advances in Neural Information Processing Systems , year=
-
[61]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=
Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2018 , publisher=
2018
-
[62]
A kernel-independent
March, William B and Xiao, Bo and Tharakan, Sameer and Yu, Chenhan D and Biros, George , booktitle=. A kernel-independent
-
[63]
Martirosyan, DM and Ohanyan, VK , journal=. On the. 2024 , publisher=
2024
-
[64]
Positivity , volume=
Positive definite metric spaces , author=. Positivity , volume=. 2013 , publisher=
2013
-
[65]
Potential Analysis , volume=
Magnitude, diversity, capacities, and dimensions of metric spaces , author=. Potential Analysis , volume=
-
[66]
A short proof of
Munthe-Kaas, Hans Z and Verdier, Olivier and Vilmart, Gilles , journal=. A short proof of
-
[67]
1981 , publisher =
The Cannonball Run , author =. 1981 , publisher =
1981
-
[68]
Quadratic programming with one negative eigenvalue is
Pardalos, Panos M and Vavasis, Stephen A , journal=. Quadratic programming with one negative eigenvalue is. 1991 , publisher=
1991
-
[69]
1972 , publisher=
Positive Definite Kernels, Continuous Tensor Products, and Central Limit Theorems of Probability Theory , author=. 1972 , publisher=
1972
-
[70]
Measuring diversity from dissimilarities with
Pavoine, Sandrine and Ollier, S. Measuring diversity from dissimilarities with. Theoretical Population Biology , volume=. 2005 , publisher=
2005
-
[71]
International Conference on Evolutionary Multi-Criterion Optimization , pages=
Comparative Analysis of Indicators for Multi-objective Diversity Optimization , author=. International Conference on Evolutionary Multi-Criterion Optimization , pages=. 2025 , organization=
2025
-
[72]
2007 , publisher=
Numerical Recipes, 3rd edition , author=. 2007 , publisher=
2007
-
[73]
Reviews of Modern Physics , volume=
Ultrametricity for physicists , author=. Reviews of Modern Physics , volume=. 1986 , publisher=
1986
-
[74]
Inequalities in Statistics and Probability , volume=
Convexity properties of entropy functions and analysis of diversity , author=. Inequalities in Statistics and Probability , volume=. 1984 , publisher=
1984
-
[75]
Enhanced multi-objective
Ren, Zhongqiang and Zhan, Richard and Rathinam, Sivakumar and Likhachev, Maxim and Choset, Howie , booktitle=. Enhanced multi-objective. 2022 , url=
2022
-
[76]
IEEE Transactions on Circuits and Systems , volume=
A new algorithm for computing a single root of a real continuous function , author=. IEEE Transactions on Circuits and Systems , volume=. 1979 , publisher=
1979
-
[77]
The climate of the
Rubel, Franz and Brugger, Katharina and Haslinger, Klaus and Auer, Ingeborg , journal=. The climate of the. 2017 , publisher=
2017
-
[78]
SIAM Journal on Computing , volume=
Computationally related problems , author=. SIAM Journal on Computing , volume=. 1974 , publisher=
1974
-
[79]
International Conference on Military Communications and Information Systems (ICMCIS), ser
Practical parameter tuning of rf jammers by optimization of expensive black-box functions , author=. International Conference on Military Communications and Information Systems (ICMCIS), ser. STO-MP-IST-200 , year=
-
[80]
On certain metric spaces arising from
Schoenberg, Isaac J , journal=. On certain metric spaces arising from. 1937 , publisher=
1937
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.