Pith. sign in

REVIEW 4 major objections 5 minor 29 references

A general framework for adaptive nonparametric dimensionality reduction

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single data-driven rule for neighborhood sizes can replace manual tuning in nearest-neighbor dimensionality reduction methods.

desk verdict Useful plug-in idea with real promise, but the 'optimal tuning' claim outruns the experiments: the evaluation confounds d* with k* and never shows that uniformity-optimal neighborhoods are reconstruction-optimal. read the letter →

arxiv 2511.09486 v2 pith:JACSIV7N submitted 2025-11-12 stat.ML cs.AIcs.LGstat.ME

Antonio Di Noia , Federico Ravenda , Antonietta Mira This is my paper · ORCID
classification stat.MLcs.AIcs.LGstat.ME
keywords dimensionalityreductionintrinsicdimensionestimationlocallyadaptiveneighborhoodsnearestneighborshyperparametertuninglinearembeddingspectralclusteringUMAP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the two most fragile hyperparameters in nearest-neighbor-based dimensionality reduction—the number of neighbors defining each local neighborhood and the dimension of the target space—can be set automatically by a joint estimator of intrinsic dimension. The estimator, ABIDE, assigns each data point its own neighbor count k*, chosen so the sampling distribution in that neighborhood is approximately uniform, and returns the overall intrinsic dimension d*. Feeding these per-point values into LLE, spectral clustering, and UMAP yields embeddings that beat default configurations on clustering metrics on MNIST, news articles, and simulated manifolds, and improves downstream classification. If correct, this gives practitioners a principled substitute for grid search in unsupervised settings where labels are absent.

What carries the argument

ABIDE (Adaptive Binomial ID Estimator): a likelihood-based estimator that jointly estimates intrinsic dimension d and per-point optimal neighbor counts k*_i. The key mechanism is a likelihood-ratio test on the spacings between successive nearest-neighbor distances; k*_i is the smallest k at which the test rejects the null that the local Poisson intensity is constant, so the chosen neighborhood is the largest one that still looks uniformly sampled. The paper's framework treats this k*_i as the 'optimal locally adaptive neighborhood structure' and uses it to build the adjacency matrix of any NN-based projection method, with d* fixing the projection dimension.

What would settle it

Take a dataset with known manifold structure and deliberately non-uniform sampling density. Run LLE* and, for every fixed k in a wide grid, run LLE with n_components = d* and evaluate a downstream clustering metric with ground-truth labels. If any fixed k (or a per-point k chosen by a different criterion, e.g., minimizing local reconstruction error) beats LLE* by a substantial margin on most such datasets, the claim that ABIDE's k* is optimal for dimensionality reduction is falsified.

Watch

Extended reading notes

Core claim

The central claim is that optimal hyperparameters for NN-based dimensionality reduction are not a single global pair (n_neighbors, n_components) but a per-point neighborhood structure. For each point x_i, ABIDE grows a ball until a likelihood-ratio test detects a break in the local homogeneity of the underlying Poisson process; the largest k before the break, k*_i, is declared the optimal neighborhood, and the binomial-count estimator at convergence yields d*. The paper shows that plugging k*_1:n and d* into LLE, spectral clustering, and UMAP—replacing fixed k and hand-set dimension—produces lower-dimensional representations that cluster more cleanly and classify more accurately than default

Load-bearing premise

The load-bearing premise is that a neighborhood in which the sampling process looks locally uniform is the right neighborhood for faithful dimensionality reduction—if uniform sampling does not coincide with low reconstruction error, the 'optimal' tuning is optimal only for uniformity, not for the embedding.

Editorial extensions

If this is right

  • Replacing fixed n_neighbors with per-point k*_i improves LLE, spectral clustering, and UMAP on clustering metrics across MNIST, news articles, and simulated manifolds.
  • The framework removes the need for grid search or manual hyperparameter choice in unsupervised dimensionality reduction, where no labels exist to validate a choice.
  • The gains are largest when data density is non-uniform; on roughly uniform data, a fixed neighborhood near the median of the k* distribution performs comparably, but the adaptive version is safer because density is unknown.
  • An out-of-sample extension projects new points by estimating their own k* via the trained d, enabling supervised use of the adaptive embeddings.
  • Any NN-based dimensionality reduction algorithm—the paper mentions DBSCAN, label propagation, and k-NN as further candidates—can inherit the same adaptive neighborhood structure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper defines 'optimal' by statistical uniformity of sampling, not by any direct measure of embedding quality; whether uniform neighborhoods also minimize reconstruction error or downstream task error is an empirical assumption worth testing head-to-head against oracle-tuned k.
  • The spread of the k* distribution itself could serve as a diagnostic: a wide spread flags strongly non-uniform density, where fixed-k methods are most unreliable and adaptive structure matters most.
  • A natural stress test is to compare LLE* against LLE with a grid of fixed k values chosen by an oracle (best ARI on test labels) on the same datasets; the paper's grid comparisons suggest LLE* is near or above average, but an oracle comparison would quantify the remaining gap.
  • The same k*_i could be plugged into density-based clustering (DBSCAN) or graph-based label propagation, turning this from a dimensionality-reduction tuning trick into a general local-bandwidth selection rule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a general framework for adaptive nonparametric dimensionality reduction: it uses ABIDE, an intrinsic-dimension estimator that also returns a per-point optimal number of nearest neighbours k*_i, to set the neighbourhood size and projection dimensionality of NN-based DR algorithms. The authors instantiate this for LLE (LLE*), spectral clustering (SC*), and UMAP (UMAP*), and evaluate the resulting embeddings through k-means clustering metrics on MNIST, News Articles, synthetic manifolds, and Iris, plus a supervised logistic-regression experiment on the embedded representations. The central claim is that ABIDE's k* and d* provide an 'optimal hyper-parameter tuning' for any DR method that relies on local neighbourhoods, and that the adaptive versions outperform default implementations on the tested benchmarks.

Significance. If the claim were fully established, the framework would be a practically useful replacement for manual tuning of n_neighbours and n_components in several common DR methods, and the connection between intrinsic-dimension estimation and neighbourhood selection is conceptually appealing. The paper rightly capitalises on ABIDE's consistency result from prior work, and it reports a broad set of experiments across four datasets and three downstream tasks, with computational overhead quantified in Table 1. The proof-of-concept extensions to SC and UMAP are useful. However, the evidence as presented does not support the word 'optimal': k*_i is selected for local Poisson-intensity homogeneity, not for any DR objective, and the empirical comparisons do not separate the effect of neighbourhood adaptivity from that of using a higher-dimensional projection. The paper is a reasonable candidate for publication after substantial revision, but the current overclaim and the confounded experimental design are load-bearing.

major comments (4)
  1. [§2.1, Eq. (3); §2.2, Eq. (5); §2.3, Eq. (7)] The abstract and §2.2 claim that the framework provides 'optimal hyper-parameter tuning' for NN-based DR. This is not established. The k*_i in Eq. (3) is the smallest k at which a likelihood-ratio test rejects homogeneity of the Poisson process intensity; it is optimal for a uniformity criterion, not for reconstruction error, clustering separability, or visualisation quality. For LLE, the objective in Eq. (7) depends on how well each point is linearly reconstructible from its neighbours, which is governed by manifold curvature and noise, not by sampling uniformity. Uniform-density neighbourhoods on a strongly curved manifold can be poor for LLE. The paper provides no theorem and no experiment tying the uniformity criterion to the downstream DR objective. Please either weaken the 'optimal' claim to 'adaptive heuristic' throughout, or add a direct comparison between k* and an oracle/strong
  2. [Tables 2–4; §3.2.2, Fig. 3] The main quantitative comparisons confound neighbourhood adaptivity with output dimensionality. In Tables 2–4, the adaptive methods embed into d* components (e.g., 11.56 for MNIST, 10.94 for News Articles in Table 1), while the default baselines use the standard 2D visualisation setting for LLE and UMAP; the SC comparison similarly does not hold the embedding dimension fixed. A large part of the reported improvement may simply be the benefit of using more than 2 components. Figure 3 partially addresses this for LLE by fixing n_components=d* and varying n_neighbours, but it does not cover SC or UMAP, and none of Tables 2–4 or Figure 3 report repeated runs or error bars despite k-means being stochastic. At minimum, add matched-dimension baselines (default method with n_components=d* and n_neighbours equal to the mean or median of k*) and report mean±std over multiple random initialisations
  3. [§2.1, Eq. (3); §3.2.2] The method is presented as avoiding hyper-parameter tuning, but it still contains a free significance level α in the definition of k*_i. The text itself notes that 'small values of α imply the identification of larger neighbourhoods' and calls this a 'choice of the researcher', yet no default α is stated and no sensitivity analysis is reported for any dataset. Since k*_i directly controls the neighbourhood graph, the results may depend non-negligibly on this choice. Please state the value of α used in all experiments and provide a sensitivity analysis (e.g., α ∈ {0.01, 0.05, 0.1}) for at least one dataset, or justify a principled default.
  4. [§3.3, Algorithm 1] The out-of-sample extension used for supervised learning is under-specified. Step 2 states 'Apply ABIDE: set intrinsic dimension to d*' for the augmented training-plus-test dataset, but ABIDE is a joint iterative estimator of d and k; fixing d to the training value changes the neighbourhood-selection criterion relative to the training procedure. The derivation of the test point's k*_test under this fixed-d variant is not given. In addition, Figure 5(B) compares LLE* to a grid of non-adaptive LLE configurations, but the grid uses different n_components values from d* (e.g., MNIST d*≈11.56 while the grid only goes to 12), so the supervised comparison is again not fully dimension-matched. Please clarify the out-of-sample algorithm and, if possible, compare against non-adaptive LLE with n_components=d* and n_neighbours equal to the summary statistics of k*.
minor comments (5)
  1. [§3.2.4] Typo: 'obatined' should be 'obtained'.
  2. [Table 3] Several entries in the News Articles row are run together without spaces ('SC 0.447 0.5570.6940.635'); please fix the numerical formatting.
  3. [§3.1] The GitHub repository is mentioned but no URL or permanent DOI is given; provide one for reproducibility.
  4. [§2.3] The sentence in the last paragraph of §2.3 claims that locally linear reconstructions are 'much more accurate' on uniform-sampling neighbourhoods; as noted in the major comments, this is an assertion rather than a result. Please either prove it under explicit conditions or soften it to a heuristic motivation.
  5. [§3.2.3, Table 3] The description of the SC baseline is incomplete: it is not clear what n_components or n_clusters are used for the default scikit-learn spectral clustering, or how these are matched to the adaptive version. Clarify the exact hyperparameter settings.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: ABIDE's k* and d* are estimated from local homogeneity, not from the downstream DR metrics; only semantic overreach and reliance on the authors' prior estimator keep the score above 0.

full rationale

The derivation chain is not circular. ABIDE estimates the intrinsic dimension d via the binomial likelihood in Eq. (1) and selects k*_i as the smallest k for which the local Poisson homogeneity likelihood-ratio test rejects (Eq. 3). These outputs are then plugged into LLE, spectral clustering, and UMAP via Eq. (5), without fitting k* or d* to the ARI, homogeneity, completeness, V-measure, accuracy, or F1 scores used in the experiments. Tables 2-4 and Figure 3 compare the adaptive versions against default configurations and grid searches, so the reported improvements are not forced by construction: k* and d* are not functions of the evaluation metrics. The only self-referential component is that ABIDE is the authors' own prior work (Di Noia et al., 2024), which is cited as the estimator's source and consistency proof. That is normal reliance on prior work, not circularity, since ABIDE has an independent consistency result and the present evaluation is external to its fitting. The main weakness is semantic: the paper calls the framework an 'optimal hyper-parameter tuning' method, but k* is optimal for local intensity homogeneity, and the equivalence between homogeneity-optimal neighborhoods and DR-optimal neighborhoods is asserted rather than proven. This is an overclaim or untested assumption, not a circular reduction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim does not introduce new physical or mathematical entities. The main hidden tuning knob is alpha in ABIDE; the rest of the machinery is imported from the authors' earlier estimator, whose consistency is assumed rather than re-proven.

free parameters (1)
  • Significance level alpha for ABIDE's likelihood-ratio test (Eq. 3) = not reported in experiments
    Controls how large k*_i can grow; the paper notes smaller alpha implies larger neighborhoods and that alpha is a researcher's choice, but never states the value used in Tables 2-4. This is a hand-chosen tuning knob in the proposed 'automatic' framework.
assumptions (4)
  • domain assumption Manifold hypothesis: X1:n are sampled in a small neighbourhood of a d-dimensional C1 manifold embedded in R^D.
    Stated in Section 2.1. If data do not lie near such a manifold, ABIDE's ID and k* are not meaningful.
  • domain assumption Local Poisson process homogeneity: point counts in balls follow a Poisson process with locally constant intensity.
    Section 2.1 models counts and uses the likelihood ratio test for k*; this uniformity assumption is the basis of the claimed optimal neighborhoods.
  • domain assumption ABIDE consistency and variance result from Di Noia et al. (2024) are taken as given.
    The paper relies on ABIDE's convergence and optimality properties without re-deriving them; this is prior work by the same authors.
  • domain assumption LLE's local linear reconstruction assumption holds: each point is well approximated by a weighted linear combination of its neighbours.
    Section 2.3 uses this to justify that adaptive neighborhoods improve reconstruction accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A general framework for adaptive nonparametric dimensionality reduction." pith.science (2026). https://pith.science/paper/JACSIV7N

@misc{pith2026251109486,
  author       = {Pith},
  title        = {Pith review of: A general framework for adaptive nonparametric dimensionality reduction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JACSIV7N}},
  note         = {Machine review of arXiv:2511.09486}
}
read the original abstract

Dimensionality reduction is a fundamental task in modern data science. Several projection methods specifically tailored to take into account the non-linearity of the data via local embeddings have been proposed. Such methods are often based on local neighbourhood structures and require tuning the number of neighbours that define this local structure, and the dimensionality of the lower-dimensional space onto which the data are projected. Such choices critically influence the quality of the resulting embedding. In this paper, we exploit a recently proposed intrinsic dimension estimator which also returns the optimal locally adaptive neighbourhood sizes according to some desirable criteria. In principle, this adaptive framework can be employed to perform an optimal hyper-parameter tuning of any dimensionality reduction algorithm that relies on local neighbourhood structures. Numerical experiments on both real-world and simulated datasets show that the proposed method can be used to significantly improve well-known projection methods when employed for various learning tasks, with improvements measurable through both quantitative metrics and the quality of low-dimensional visualizations.

Figures

Figures reproduced from arXiv: 2511.09486 by the authors.

Figure 1
Figure 1. Representation of the main LLE∗ steps. 3 Numerical experiments 3.1 Datasets To evaluate the effectiveness of the proposed method, various datasets with different char￾acteristics are used, allowing for an in-depth exploration across a wide spectrum of data types. In [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Data visualisation in the reduced space using LLE [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Visualisation of performance metrics of ARI, Homogeneity, Completeness, and [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualisation of the performance of the 4 considered metrics ARI, Homogeneity, [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: (A) The results of the accuracy score for the datasets considered are shown for different configurations of LLE, adaptive (LLE∗ ) and non-adaptive (LLE). (B) Box￾plots of accuracy scores comparing adaptive and non-adaptive dimensionality reduction approaches across MNI…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 4 linked inside Pith

  1. [1]

    \'A lvarez-Meza, A., Valencia-Aguirre, J., Daza-Santacoloma, G., and Castellanos-Dom \' nguez, G. (2011). Global and local choice of the number of nearest neighbors in locally linear embedding. Pattern Recognition Letters , 32(16):2171--2177

    2011

  2. [2]

    K., and Talib, R

    Ayesha, S., Hanif, M. K., and Talib, R. (2020). Overview and comparative study of dimensionality reduction techniques for high dimensional data. Information Fusion , 59:44--58

    2020

  3. [3]

    and Bengio, Y

    Bergstra, J. and Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of machine learning research , 13(2)

    2012

  4. [4]

    and Devroye, L

    Biau, G. and Devroye, L. (2015). Lectures on the nearest neighbor method , volume 246. Springer

    2015

  5. [5]

    A., D otko, P., Harvey, J., Malinowski, J., and Yim, K

    Binnie, J. A., D otko, P., Harvey, J., Malinowski, J., and Yim, K. M. (2025). A survey of dimension estimation methods. arXiv preprint arXiv:2507.13887

    arXiv 2025

  6. [6]

    Coelho, D., Madureira, A., Pereira, I., and Gon c alves, R. (2022). A review on dimensionality reduction for machine learning. In International Conference on Innovations in Bio-Inspired Computing and Applications , pages 287--296. Springer

    2022

  7. [7]

    Del Giudice, M. (2021). Effective dimensionality: A tutorial. Multivariate behavioral research , 56(3):527--542

    2021

  8. [8]

    Di Noia, A., Macocco, I., Glielmo, A., Laio, A., and Mira, A. (2024). Beyond the noise: intrinsic dimension estimation with optimal neighbourhood identification. arXiv preprint arXiv:2405.15132

    arXiv 2024

Show all 29 references
  1. [9]

    Fisher, R. A. (1936). Iris . UCI Machine Learning Repository. DOI : https://doi.org/10.24432/C56C76

    1936 doi

  2. [10]

    Ghojogh, B., Ghodsi, A., Karray, F., and Crowley, M. (2020). Locally linear embedding and its variants: Tutorial and survey. arXiv preprint arXiv:2011.10925

    2020 arXiv

  3. [11]

    and Hammer, B

    Gisbrecht, A. and Hammer, B. (2015). Data visualization by nonlinear dimensionality reduction. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , 5(2):51--73

    2015

  4. [12]

    Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of educational psychology , 24(6):417

    1933

  5. [13]

    Huang, J., Jiao, Y., Liao, X., Liu, J., and Yu, Z. (2024). Deep dimension reduction for supervised representation learning. IEEE Transactions on Information Theory

    2024

  6. [14]

    and Arabie, P

    Hubert, L. and Arabie, P. (1985). Comparing partitions. Journal of Classification , 2:193--218

    1985

  7. [15]

    M., Branicky, M

    LaValle, S. M., Branicky, M. S., and Lindemann, S. R. (2004). On the relationship between classical grid search and probabilistic roadmaps. The International Journal of Robotics Research , 23(7-8):673--692

    2004

  8. [16]

    LeCun, Y., Cortes, C., and Burges, C. (2010). Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist , 2

    2010

  9. [17]

    Lloyd, S. (1982). Least squares quantization in pcm. IEEE transactions on information theory , 28(2):129--137

    1982

  10. [18]

    McInnes, L., Healy, J., and Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426

    2018 arXiv

  11. [19]

    and Gurevych, I

    Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks

    2019

  12. [20]

    and Hirschberg, J

    Rosenberg, A. and Hirschberg, J. (2007). V -measure: A conditional entropy-based external cluster evaluation measure. In Eisner, J., editor, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning...

    2007

  13. [21]

    Roweis, S. T. and Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. science , 290(5500):2323--2326

    2000

  14. [22]

    and Tenenbaum, J

    Silva, V. and Tenenbaum, J. (2002). Global versus local methods in nonlinear dimensionality reduction. Advances in neural information processing systems , 15

    2002

  15. [23]

    Snoek, J., Larochelle, H., and Adams, R. P. (2012). Practical bayesian optimization of machine learning algorithms

    2012

  16. [24]

    B., Silva, V

    Tenenbaum, J. B., Silva, V. d., and Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. science , 290(5500):2319--2323

    2000

  17. [25]

    and Fran c ois, D

    Verleysen, M. and Fran c ois, D. (2005). The curse of dimensionality in data mining and time series prediction. In International work-conference on artificial neural networks , pages 758--770. Springer

    2005

  18. [26]

    Von Luxburg, U. (2007). A tutorial on spectral clustering. Statistics and computing , 17:395--416

    2007

  19. [27]

    Xue, J., Zhang, B., and Qiang, Q. (2023). Local linear embedding with adaptive neighbors. Pattern Recognition , 136:109205

    2023

  20. [28]

    and Shami, A

    Yang, L. and Shami, A. (2020). On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing , 415:295--316

    2020

  21. [29]

    and Wang, J

    Zhang, Z. and Wang, J. (2006). Mlle: Modified locally linear embedding using multiple weights. Advances in neural information processing systems , 19

    2006

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.