REVIEW 3 major objections 5 minor 2 cited by
GhostUMAP2: Measuring and Analyzing (r,d)-Stability of UMAP
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper defines (r,d)-stability for UMAP projections using ghost duplicates and an adaptive dropping scheme that flags stochasticity-driven point placements at roughly 2.4x lower cost.
desk verdict Useful extension of GhostUMAP with a real definition-implementation mismatch that needs fixing before the central claims can be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the ghost: a copy of a data point that carries the same high-dimensional vector and the same graph connections, but starts at a random position in a radius-r circle around the target's initial projection. The central identity is the stability distance $d_i = \max_k \| y'_i - g'_{ik} \|_2$, the largest final distance from the original point to any of its ghosts, with (r,d)-stability defined by $d_i \leq d$. The acceleration machinery is the adaptive dropping scheme, which computes an exponential moving average $D_i = \beta d_i + (1-\beta) D_i$, sets the drop threshold to the mean of $D_i$, removes ghosts of points whose $D_i$ is below that threshold, and freezes the $D_i$ values of dropped points to prevent threshold inflation. Together these pieces turn a heuristic notion of 'reliable projection' into a concrete, computable quantity that can be adjusted after the fact through d without rerunning the optimization.
What would settle it
Run UMAP many times on the same dataset, record each point's position per run, and compare the empirical per-point spread with the ghost spread from a single GhostUMAP2 run; substantial mismatch (for example, ghosts far tighter or far wider than the observed run-to-run scatter) would show that (r,d)-stability measures artificial perturbations rather than true stochastic instability.
Extended reading notes
Core claim
The central claim is that stochastic instability in UMAP can be measured pointwise, in one run, by the (r,d)-stability criterion: ghosts—duplicates of a data point sharing its high-dimensional vector but initialized at random positions inside a radius-r circle—are optimized together with the original projections, and the point is (r,d)-stable exactly when the maximum final distance between the original point and its ghosts is at most d (Eq. 8). The paper argues that because ghosts use separate negative samples and never influence the originals, their final spread reflects how much the target's placement depends on random initialization and negative sampling rather than on neighboring structure. It further claims that the adaptive dropping scheme—monitoring an exponentially smoothed version of that maximum distance and dropping ghosts whose smoothed distance falls below the running mean—finds unstable points with F1 around 0.90 while cutting runtime by up to 60 percent relative to the unoptimized baseline. Use cases on C. elegans, Fashion-MNIST, and AG News illustrate patterns where original and ghost projections split into multiple groups or where the original sits apart from a tight ghost cluster, which the paper interprets as evidence that the original location is a stochastic artifact.
Load-bearing premise
The framework's usefulness rests on the premise that ghosts—points that share the target's high-dimensional vector, start at random positions in a radius-r circle, and are optimized with separate negative samples—faithfully represent where the target would actually land across independent stochastic UMAP runs, a premise the paper does not test against genuinely repeated runs.
Editorial extensions
If this is right
- Practitioners can flag unreliable points in a single GhostUMAP2 run instead of comparing many repeated UMAP embeddings.
- The threshold d can be swept interactively after optimization, so users can choose how aggressive to be when hiding or inspecting unstable points.
- Unstable points often sit at cluster boundaries or have internally ambiguous neighborhoods, so the flags can direct attention to data that deserves a second look before downstream conclusions are drawn.
- The four observed patterns (compact convergence, split groups, wide scatter, and a separated original with a tight ghost cluster) provide concrete, interpretable signatures for why a particular point is or is not stable.
- Because the ghost mechanism is stated to extend to other force-based DR methods such as t-SNE or LargeVis, the per-point stability concept is not tied to UMAP alone.
Reading between the lines
- One direct test the paper leaves open: compare the ghost spread from a single GhostUMAP2 run with the empirical spread of each point's position across many actual UMAP runs; if the two distributions diverge, the (r,d) measure would be quantifying artificial perturbations rather than true run-to-run variability.
- The observed weak correlation between r and final ghost distance suggests the initial ball is distorted non-linearly during optimization, so a single farthest-ghost statistic may under-represent multimodal alternatives; a clustering or density summary of ghost positions could offer a richer stability signal.
- If ghosts do track genuine stochastic variability, (r,d)-stability could be used as a per-point weight in downstream analyses, down-weighting or masking unstable points before clustering or annotation rather than only highlighting them for visual inspection.
- The adaptive dropping idea is not specific to ghosts: the same early-exit logic (compute a cheap per-point uncertainty proxy, drop the confident ones, freeze their values) could accelerate other stochastic embedding algorithms that need per-point reliability estimates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces GhostUMAP2, a framework for measuring pointwise stability of UMAP projections against stochasticity from random initialization and negative sampling. The authors define (r,d)-stability: a point is stable if its 'ghosts'—duplicate high-dimensional vectors initialized in a radius-r circle around the point's initial projection and co-optimized with separate negative samples—remain within radius d of the point's final projection. To reduce the computational cost of tracking many ghosts, the paper proposes an adaptive dropping scheme that discards ghosts of points deemed stable early in optimization, and it reports runtime speedups (about 2.4x over no-reduction baseline) while maintaining F1 scores around 0.90. The paper also presents GhostExplorer, an interactive visualization tool, and usage guidelines based on the observed instability patterns. The central claim is that GhostUMAP2 provides an efficient, actionable measure of how much each point's final position is determined by stochasticity rather than by neighboring structure.
Significance. If the ghost-based measure faithfully captures run-to-run variability of UMAP projections, the paper would fill a genuine gap: existing DR quality measures focus on distortion relative to the high-dimensional space, not on stochastic instability. The framework is conceptually clear, and the adaptive dropping scheme is a practical contribution with plausible engineering value. The paper is also honest in acknowledging limitations, such as not addressing unstable points actively and not extending to other DR techniques. However, the significance is currently undercut by two internal inconsistencies between the formal definition and the implementation, and by an evaluation design that does not validate the core premise that ghosts represent actual stochastic outcomes. The reported high F1 scores, in particular, should be interpreted as measuring consistency between the adaptive and no-reduction variants of the same ghost-based procedure, not as evidence that the method detects true run-to-run instability.
major comments (3)
- [Sec. 3.2, Eq. (8); Sec. 3.4, Alg. 4; Sec. 3.3, Alg. 1] The formal definition and the implementation compute different quantities. Eq. (8) defines d_i as the maximum Euclidean distance over all M ghosts, but Alg. 4 (line 3) sets j = ceil(M * sensitivity), and with the default sensitivity = 0.9 and M = 16, it uses the 15th-farthest ghost, i.e., the 90th percentile, not the maximum. The paper states in Sec. 3.4 that sensitivity = 1.0 recovers the farthest ghost, which confirms the default does not match Eq. (8). Separately, Eq. (6) initializes ghosts around the initial projection y_i, but Alg. 1 (lines 8–9) only creates ghosts after e >= ceil(n_epochs * lazy_gen), with default lazy_gen = 0.2; for MNIST this means ghosts are spawned at epoch 100 of 500, well after the optimization has substantially moved the projection. Consequently, the quantity actually evaluated in Sec. 4 (and used in Fig. 5 and the usage guidelines) is not the (r,d)-stability defined in Sec. 3.2. The authors should either change the definition to match the implemented percentile-based distance and delayed generation, or change the implementation (e.g., set sensitivity = 1.0 and lazy_gen = 0.0) and rerun the benchmarks.
- [Sec. 4.1, 'Measures' and 'Result and Discussion'] The F1 evaluation is circular with respect to the central claim. The ground truth is defined as the set of points whose d_i > 0.1 under the 'no reduction' variant of GhostUMAP2—that is, the same ghost simulation, the same Eq. (8) distance (or its percentile-based implementation), and the same optimization procedure. The predicted set for the adaptive method is the set of points that survive the dropping scheme and have d_i > 0.1. Thus, the reported F1 score around 0.90 primarily measures the internal consistency of the dropping approximation against the no-reduction computation; it does not measure whether either the ground truth or the prediction corresponds to actual instability of UMAP across independent runs. The paper would need an external validation—for example, comparing ghost-based unstable points against points whose positions vary most across multiple UMAP runs with different random seeds—before claiming the framework 'maintains unstable points' in a meaningful sense.
- [Sec. 3.3, 'We set three requirements...' and Sec. 7.2] The load-bearing premise that ghosts faithfully represent the range of positions a point would occupy across genuine stochastic UMAP runs is untested. The paper states as a requirement that 'ghost projections must accurately represent their corresponding projections of the original points,' but no experiment compares the distribution of ghost projections with the distribution of the same point's final positions across repeated UMAP runs with different initial embeddings and negative sampling. Without such a comparison, the (r,d)-stability measure could be quantifying sensitivity to the artificial perturbation of the ghost simulation rather than to the actual stochasticity of UMAP. The authors should add a validation experiment, perhaps measuring the overlap between ghost-confined circles and the empirical spread of repeated runs, or the correlation between d_i and run-to-run displacement.
minor comments (5)
- [Sec. 4.1, 'Measures'] There is a typo in the sentence defining the predicted set: 'points that have ghsots' should read 'points that have ghosts.'
- [Sec. 1, abstract and Sec. 4.1] The abstract claims 'up to 60%' speedup while Sec. 4.1 reports a 2.4x speedup over no-reduction, which corresponds to about 58% reduction in runtime; the wording should be made consistent to avoid overstating the speedup.
- [Sec. 3.2, Eq. (8) context] The phrase 'We assume both Y and Y' are normalized into a range [[0,1], [0,1]]' has malformed bracket notation; it should be '[0,1] x [0,1]' or similar.
- [Sec. 4.2, Table 3] In Table 3, the row for M=0 lists 'Time (s)' values for UMAP but the header says 'M'; this is understandable but should be clarified, for example by adding a footnote that M=0 corresponds to plain UMAP.
- [Sec. 5.1] The statement 'Increasing its value leads to a higher number of unstable points' appears to be a typo; since d is the stability threshold, increasing d should lead to fewer unstable points. This should be corrected to avoid confusion.
Circularity Check
No significant circularity: the (r,d)-stability definition, ghost optimization, and adaptive-dropping benchmark are self-contained, and the reported F1 is an internal-consistency check of the dropping approximation rather than a derivation that reduces to its own inputs.
full rationale
GhostUMAP2's central chain is definitional and self-contained. Section 3.2 defines d_i as the maximum ghost distance in Eq. (8) and (r,d)-stability as d_i <= d; Section 3.3 specifies ghost generation (Eq. 6), joint optimization (Eq. 7), and the explicit requirements that ghosts not affect original projections, share the target's high-dimensional vector, and use separate negative samples. Section 3.4 presents adaptive dropping as an approximation, and Section 4 evaluates it via F1 against the no-reduction run. Although the ground-truth and predicted unstable sets are both computed from ghost simulations using the same instability definition, that is the appropriate way to measure whether dropping preserves the unstable set; the predicted set is not derived from the ground-truth labels, and no fitted parameter is renamed as a prediction. The self-citations to [14] describe prior GhostUMAP and the halving baseline, but they are not load-bearing for the new definition or the adaptive-dropping algorithm, and no uniqueness theorem or ansatz is imported from the authors' prior work. The skeptical observation that Alg. 4 uses the j-th farthest ghost distance with sensitivity=0.9 while Eq. 8 defines d_i as the maximum, and that lazy_gen=0.2 spawns ghosts after 20% of epochs rather than at the initial projection, is a definition-implementation mismatch and a threat to construct validity, not a circularity: the experiments may validate an unnamed quantity, but they do not reduce Eq. (8) to its own input. No circular step meeting the quoted-reduction standard is present.
Assumptions & free parameters
free parameters (7)
- r =
0.1 (default, user-specified)
- d =
0.1 (default, user-adjustable without recomputation)
- M =
16 (default)
- lazy_gen =
0.2 (default)
- drop_start =
0.4 (default)
- beta =
0.2 (default)
- sensitivity =
0.9 (default)
assumptions (5)
- ad hoc to paper Ghosts initialized by sampling within a circle of radius r and optimized independently with separate negative samples faithfully simulate the stochastic outcomes for a point's final projection.
- domain assumption Race conditions from Numba parallelization are negligible as a source of projection instability.
- domain assumption UMAP's cross-entropy loss and force equations (Eqs. 3-5) are a valid model of the layout optimization.
- domain assumption Initial and final projections are normalized to [0,1]^2 before distances are measured.
- ad hoc to paper Mean of EMA-smoothed ghost distances is a suitable global threshold for dropping stable points.
invented entities (1)
-
Ghost projections (ghosts)
Cite this review
Pith. "Pith review of GhostUMAP2: Measuring and Analyzing (r,d)-Stability of UMAP." pith.science (2026). https://pith.science/paper/UZQPQQRE
@misc{pith2026250717174,
author = {Pith},
title = {Pith review of: GhostUMAP2: Measuring and Analyzing (r,d)-Stability of UMAP},
year = {2026},
howpublished = {\url{https://pith.science/paper/UZQPQQRE}},
note = {Machine review of arXiv:2507.17174}
}
read the original abstract
Despite the widespread use of Uniform Manifold Approximation and Projection (UMAP), the impact of its stochastic optimization process on the results remains underexplored. We observed that it often produces unstable results where the projections of data points are determined mostly by chance rather than reflecting neighboring structures. To address this limitation, we introduce (r,d)-stability to UMAP: a framework that analyzes the stochastic positioning of data points in the projection space. To assess how stochastic elements, specifically initial projection positions and negative sampling, impact UMAP results, we introduce "ghosts", or duplicates of data points representing potential positional variations due to stochasticity. We define a data point's projection as (r,d)-stable if its ghosts perturbed within a circle of radius r in the initial projection remain confined within a circle of radius d for their final positions. To efficiently compute the ghost projections, we develop an adaptive dropping scheme that reduces a runtime up to 60% compared to an unoptimized baseline while maintaining approximately 90% of unstable points. We also present a visualization tool that supports the interactive exploration of the (r,d)-stability of data points. Finally, we demonstrate the effectiveness of our framework by examining the stability of projections of real-world datasets and present usage guidelines for the effective use of our framework.
Figures
Figures from the paper (4 more)
Forward citations
Cited by 2 Pith papers
-
When One Point Is Not Enough: Addressing Ambiguous Instances in Dimensionality Reduction by Splitting
Data points genuinely similar to multiple dissimilar neighborhoods are detected as local articulation points of a sparsified high-dimensional graph and split into multiple projected copies, one per neighborhood.
-
FlexMUSE: Multimodal Unification and Semantics Enhancement Framework with Flexible interaction for Creative Writing
FlexMUSE, a claimed multimodal creative-writing framework and its ArtMUSE dataset, are unsupported because the submitted full text is an unrelated dimensionality-reduction paper (UMATO).
Reference graph
Works this paper leans on
- [1]
-
[2]
M. Belkin and P. Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Information Processing Systems, 14, 2001. 7 pages. https://proceedings.neurips.cc/paper_files/ paper/2001/file/f106b7f99d2cb30c3db1c3cc0fde9ccb-Paper.pdf. 1
work page 2001
-
[3]
R. G. Brown. Exponential Smoothing for Predicting Demand . Philip Morris Records; Master Settlement, 1956. https://www.industrydocuments. ucsf.edu/docs/jzlc0130. 4, 7
work page 1956
-
[4]
T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha. Deep learning for classical Japanese literature. 2018. 8 pages. doi: 10.20676/00000341 6
-
[5]
M. Espadoto, R. M. Martins, A. Kerren, N. S. Hirata, and A. C. Telea. Toward a quantitative survey of dimension reduction techniques. IEEE Transactions on Visualization and Computer Graphics, 27(3):2153–2173,
-
[6]
T. Fujiwara, Y .-H. Kuo, A. Ynnerman, and K.-L. Ma. Feature learn- ing for nonlinear dimensionality reduction toward maximal extraction of hidden patterns. In Proc. PacificVis, pp. 122–131, 2023. doi: 10.1109/ PacificVis56936.2023.00021 2
-
[7]
G. E. Hinton and S. Roweis. Stochastic neighbor embed- ding. Advances in Neural Information Processing Systems , 15,
-
[8]
H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6):417, 1933. doi: 10.1037/h0071325 2
doi:10.1037/h0071325 1933
Show all 41 references
-
[10]
Jeon, H.-K
H. Jeon, H.-K. Ko, J. Jo, Y . Kim, and J. Seo. Measuring and explaining the inter-cluster reliability of multidimensional projections. IEEE Trans- actions on Visualization and Computer Graphics, 28(1):551–561, 2022. doi: 10.1109/TVCG.2021.3114833 2
2022
-
[11]
Jeon, H.-K
H. Jeon, H.-K. Ko, S. Lee, J. Jo, and J. Seo. Uniform manifold approxima- tion with two-phase optimization. In Proc. VIS, pp. 80–84. IEEE, 2022. doi: 10.1109/VIS54862.2022.00025 5
2022
-
[12]
J. Jo, F. Vernier, P. Dragicevic, and J.-D. Fekete. A declarative rendering model for multiclass density maps. IEEE Transactions on Visualization and Computer Graphics, 25(1):470–480, 2019. doi: 10.1109/TVCG.2018. 2865141 5
2019 doi
-
[13]
M. Jung, J. Choi, and J. Jo. Projection Ensemble: Visualizing the robust structures of multidimensional projections. In Proc. VIS, pp. 46–50. IEEE,
-
[14]
M. Jung, T. Fujiwara, and J. Jo. GhostUMAP: Measuring pointwise instability in dimensionality reduction. In Proc. VIS, pp. 161–165. IEEE,
-
[15]
J. B. Kruskal. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1):1–27, 1964. doi: 10. 1007/BF02289565 2
1964
-
[16]
S. K. Lam, A. Pitrou, and S. Seibert. Numba: A LLVM-based Python JIT compiler. In Proc. LLVM, pp. 1–6, 2015. doi: 10.1145/2833157.2833162 1, 2
2015
-
[17]
LeCun, C
Y . LeCun, C. Cortes, and C. J.C. Burges. The MNIST database of hand- written digits. https://www.openml.org/search?type=data&id=554, 1999. Accessed: 2024-3-26. 6
1999
-
[18]
J. A. Lee and M. Verleysen. Quality assessment of dimensionality reduc- tion: Rank-based criteria. Neurocomputing, 72(7-9):1431–1443, 2009. doi: 10.1016/j.neucom.2008.12.017 2
2009 doi
-
[19]
Lespinats and M
S. Lespinats and M. Aupetit. CheckViz: Sanity check and topological clues for linear and non-linear mappings. Computer Graphics Forum, 30(1):113–125, 2011. doi: 10.1111/j.1467-8659.2010.01835.x 2
2011
-
[20]
H.-Y . Lu, T. Fujiwara, M.-Y . Chang, Y .-c. Fu, A. Ynnerman, and K.-L. Ma. Visual analytics of multivariate networks with representation learning and composite variable construction. IEEE Transactions on Visualization and Computer Graphics, 2024 (Early Access). doi: 10.1109/T...
2024 doi
-
[21]
Mayorga and M
A. Mayorga and M. Gleicher. Splatterplots: Overcoming overdraw in scatter plots. IEEE Transactions on Visualization and Computer Graphics, 19(9):1526–1538, 2013. doi: 10.1109/TVCG.2013.65 5
2013 doi
- [22]
-
[23]
Mikolov, I
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their composi- tionality. Advances in Neural Information Processing Systems , 26,
-
[24]
L. G. Nonato and M. Aupetit. Multidimensional projection for visual ana- lytics: Linking techniques with distortions, tasks, and layout enrichment. IEEE Transactions on Visualization and Computer Graphics, 25(8):2650– 2673, 2018. doi: 10.1109/TVCG.2018.2846735 2
2018
-
[25]
J. S. Packer, Q. Zhu, C. Huynh, P. Sivaramakrishnan, E. Preston, H. Dueck, D. Stefanik, K. Tan, C. Trapnell, J. Kim, et al. A lineage-resolved molecu- lar atlas of C. elegans embryogenesis at single-cell resolution. Science, 365(6459):eaax1971, 2019. doi: 10.1126/science.aax1971 6, 8
2019 doi
-
[26]
F. V . Paulovich, L. G. Nonato, R. Minghim, and H. Levkowitz. Least square projection: A fast high-precision multidimensional projection tech- nique and its application to document mapping. IEEE Transactions on Visualization and Computer Graphics, 14(3):564–575, 2008. doi: 10....
2008
-
[27]
J. Rennie. 20 Newsgroups. http://qwone.com/~jason/20Newsgroups/. Accessed: 2024-04-30. 6
2024
- [28]
-
[29]
M. Sips, B. Neubert, J. P. Lewis, and P. Hanrahan. Selecting good views of high-dimensional data using class consistency. Computer Graphics Forum, 28(3):831–838, 2009. doi: 10.1111/j.1467-8659.2009.01467.x 2
2009
-
[30]
Stahnke, M
J. Stahnke, M. Dörk, B. Müller, and A. Thom. Probing projections: Interac- tion techniques for interpreting arrangements and errors of dimensionality reductions. IEEE Transactions on Visualization and Computer Graphics, 22(1):629–638, 2016. doi: 10.1109/TVCG.2015.2467717 2
2016
-
[31]
J. Tang, J. Liu, M. Zhang, and Q. Mei. Visualizing large-scale and high- dimensional data. In Proc. WWW, pp. 287–297, 2016. doi: 10.1145/ 2872427.2883041 1, 5, 9
2016
-
[32]
J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei. LINE: Large- scale information network embedding. In Proc. WWW, pp. 1067–1077,
-
[33]
Venna and S
J. Venna and S. Kaski. Local multidimensional scaling. Neural Networks, 19(6-7):889–899, 2006. doi: 10.1016/j.neunet.2006.05.014 2
2006 doi
-
[34]
H. Xiao, K. Rasul, and R. V ollgraf. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747,
-
[35]
Zhang, J
X. Zhang, J. Zhao, and Y . LeCun. Character-level convolutional networks for text classification. Advances in Neural Information Processing Systems, 28, 2015. 9 pages. https://proceedings.neurips.cc/paper_files/paper/2015/ file/250cf8b51c773f3f8dc8b4be867a9a02-Paper.pdf. 6
2015
-
[2002]
https://proceedings.neurips.cc/paper_files/paper/2002/file/ 6150ccc6069bea6b5716254057a194ef-Paper.pdf
8 pages. https://proceedings.neurips.cc/paper_files/paper/2002/file/ 6150ccc6069bea6b5716254057a194ef-Paper.pdf. 2
2002
-
[2013]
https://proceedings.neurips.cc/paper_files/paper/2013/file/ 9aa42b31882ec039965f3c4923ce901b-Paper.pdf
9 pages. https://proceedings.neurips.cc/paper_files/paper/2013/file/ 9aa42b31882ec039965f3c4923ce901b-Paper.pdf. 1
2013
-
[2015]
doi: 10.1145/2736277.274109 1
- [2017]
-
[2019]
doi: 10.1109/TVCG.2019.2944182 2
2019
-
[2023]
doi: 10.1109/VIS54172.2023.00018 2
2023
-
[2024]
doi: 10.1109/VIS55277.2024.00040 1, 4, 5, 6, 9
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.