REVIEW 3 major objections 5 minor 33 references
Navigating High-Dimensional Backstage: A Guide for Exploring Literature for the Reliable Use of Dimensionality Reduction
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that a six-question self-check can route any analyst through the literature on reliable dimensionality-reduction visual analytics and into the papers that match their expertise.
desk verdict A clearly presented, honest design contribution that builds a reading guide on the authors' own survey; the value is real but the self-assessment routing is unvalidated, and the three-expert evaluation doesn't reach the novices it targets. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the six-item checklist I1–I6 understood as a routing device rather than a quiz: affirmative answers are the exit condition that moves a reader forward, and each negative answer names a paper cluster to read before proceeding. The paper's own reference taxonomy is the second half of the machinery — the six classes (Pioneer, Judge, Instructor, Explorer, Explainer, Architect) taken from the authors' prior survey, which organize the otherwise sprawling reliability literature into stable destinations that the checklist can point at. A web flow diagram operationalizes the routing, turning the taxonomy into a clickable path that a reader follows in roughly increasing order of expertise.
What would settle it
Give a mixed cohort of DR users the six-item checklist plus an independent, objective DR-knowledge exam, and compare the checklist routing against exam performance; the central mechanism collapses if a substantial share of low-scoring respondents answer yes to the early items, or high-scoring respondents answer no, because the guide would systematically misroute exactly the readers it claims to serve. A second check is a controlled study in which novice groups follow the guide versus the raw survey, then complete the same DR-reliability comprehension test, which would settle whether the ordered reading path actually produces the expertise gain claimed.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a survey can be converted into an instrument: the six-stage checklist I1–I6, each stage a yes/no question, with every 'no' answer pointing to the cluster of papers that supplies the missing competence. I1 routes novices to canonical introductory texts on t-SNE, UMAP, PCA, autoencoders, and DR library guides; I2 sends those unfamiliar with technique diversity to static Pioneer papers; I3 sends those unsure of task-to-technique optimality to Instructor benchmark and comparison papers; I4 sends those unaware of projection error to Judge papers and distortion-type Explainer papers; I5 routes to Explorer papers on subspace analysis; and I6 routes to Architect papers, attribute-type Explainers, and interactive Pioneers for interpretability and domain-knowledge integration. The paper's evidence that this works is an interview study in which three expert researchers independently rate the guide's significance, comprehensiveness, and usefulness, with average ratings above four on a five-point scale for most questions. The authors position the guide as the actionable counterpart to their own comprehensive survey, and explicitly open the door to building similar guides from other surveys.
Load-bearing premise
The guide assumes that readers answer the six self-assessment questions honestly and correctly, so that a yes/no answer reflects real expertise and routes each reader to the right paper cluster; a secondary premise is that the six-class taxonomy from the authors' own survey, on which the routing map is built, remains a valid picture of the literature.
Editorial extensions
If this is right
- A reader who works through the checklist in order gains the prerequisite knowledge — technique basics, technique diversity, and technique comparison — before confronting advanced reliability topics such as evaluation and interaction.
- A practitioner in any domain can locate the specific literature that addresses their context: evaluation metrics for projection accuracy, distortion awareness, subspace exploration, interpretability, or interactive refinement.
- The guide lowers the entry barrier for analysts outside visualization research, such as bioinformatics or business users, who apply DR but lack the visual-literacy background to navigate the literature alone.
- If the interview findings are representative, experts judge the guide as significant, comprehensive, and useful, meaning it can plausibly serve as a reading-course skeleton for newcomers.
- The survey-to-guide pattern is portable: other surveys in visualization could be converted the same way, including combining several surveys into a single unified guide.
Reading between the lines
- Because the checklist relies on self-reported expertise, a miscounting reader can be silently misrouted; a natural extension the paper does not run is a validation study pairing the checklist with an objective DR-knowledge quiz to measure routing accuracy.
- The guide implicitly asserts a reading order (prerequisites before advanced topics); a testable prediction is that novices who follow I1–I3 before I4–I6 will understand the advanced papers better than novices who start with evaluation or interaction papers.
- Each recommended cluster still contains dozens of papers, so the next useful refinement — suggested by the interviewed experts — is tagging papers by difficulty and by concrete analytic task, producing finer-grained recommendations.
- The taxonomy itself has a shelf life: as new DR reliability methods and metrics appear, both the classification and the per-item reading lists would need periodic re-curation to keep the routing valid.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a reading guide for the literature on reliable dimensionality reduction (DR) based visual analytics. The guide consists of a six-item checklist (I1–I6) that a practitioner answers yes/no, and a flow diagram that routes the reader to one or more paper clusters (Pioneer, Judge, Instructor, Explorer, Explainer, Architect) drawn from the authors' prior survey [JLK*25]. The central claims are that the guide helps practitioners assess their current DR expertise and identify papers that will enhance their understanding. The paper evaluates the guide through semi-structured interviews with three DR/visualization experts, who rated its significance, comprehensiveness, and usefulness on a Likert scale. The reported results are generally positive, with one usefulness item (Q6) averaging 3.66, below the agreement threshold. The authors acknowledge limitations, including the reliability of self-reported expertise and the coarse granularity of recommendations.
Significance. If the guide works as intended, it would fill a real gap: novices are often overwhelmed by the large and fragmented DR reliability literature, and existing surveys do not provide actionable reading paths. The paper has a clear, well-structured artifact (checklist + flow diagram + web guide) that is easy to understand and potentially useful. The authors are transparent about their process and limitations, and they identify sensible future directions. However, the significance is conditional on the validity of the routing mechanism, which is not directly tested. The expert interviews provide weak evidence for the central claim: they capture expert opinion about the guide's design, not whether actual practitioners can use the checklist to accurately self-assess and to improve their DR knowledge. The small sample (n=3) further limits the strength of the conclusions. The paper is honest about these constraints in Section 6.2, but the Abstract and Conclusion phrase the contributions as validated rather than as a promising but unproven approach.
major comments (3)
- [Section 5 and Table 2] The evaluation does not test the central mechanism of the guide. The core claim is that the I1–I6 checklist helps practitioners assess their expertise and that the recommended reading improves their understanding. The expert interview only solicits opinions on significance, comprehensiveness, and usefulness; it does not measure whether a practitioner's yes/no self-reports correspond to actual knowledge, whether the flow diagram routes readers to appropriate papers, or whether following the guide yields learning gains. In particular, Q6 ('Following the guide will help analysts properly use DR in different analytical contexts') averaged 3.66, which is below the 'agree' threshold (4), yet the paper concludes that the experts 'validate' usefulness. The evidence is therefore insufficient to support the Abstract's claim that the guide helps practitioners assess expertise and identify beneficial papers. I recommend either adding a user study with target practitioners (e.g., measuring agreement between checklist routing and an objective knowledge probe, or measuring learning before/after reading selected papers) or substantially softening the claims to state that expert feedback suggests the guide is promising.
- [Section 4 and Section 6.2] The checklist's routing signal is unvalidated self-reported expertise. The guide's entire added value over the prior survey [JLK*25] lies in the yes/no questions I1–I6; the flow diagram sends readers to different paper clusters based solely on these answers. The authors themselves note in Section 6.2 that self-reports 'may be less reliable than probing their actual knowledge.' This is not a minor caveat: the target users are novices, who are least able to calibrate their own competence. For example, a novice who overestimates at I2 ('I am familiar with diverse DR techniques beyond t-SNE, UMAP, and PCA') would skip foundational Pioneer material, while underestimation leads to wasted effort. No data are provided on misrouting rates, agreement with objective expertise measures, or downstream learning outcomes. Because the routing mechanism is the paper's main contribution, this gap is load-bearing. I suggest either adding a small validation study (e.g., comparing self-reports with a short quiz) or explicitly reframing the guide as an opinionated heuristic that has not yet been empirically validated.
- [Section 5.1] The participant sample is small and the participants are not the guide's target audience. Three experts with Ph.D.s and publication records in DR visualization are exactly the people least likely to need the guide, and their judgments about usefulness for novices are speculative. With n=3, the quantitative claims 'experts agree' are fragile: a single score change of 1 point on several items would flip the 'agreement' interpretation. The paper should report the results as preliminary expert feedback and discuss the limits of this evidence, rather than as validation of the guide. A complementary evaluation with novice analysts or with a larger, more representative sample is needed to support the paper's claims.
minor comments (5)
- [Section 3] Typo: 'hree authors' should be 'Three authors' in the Procedure paragraph.
- [Section 2, Architect description] The phrase 'inaccuracy of instability' appears to be a typo; it should likely be 'inaccuracy or instability' or similar.
- [Section 4 and Figure 1] The text for I4 says the Judge and Explainer (distortion) groups are recommended, and I6 recommends Architect, Explainer (attribute), and Pioneer (interactive). The flow diagram appears to align with this, but the mapping is not explicitly annotated in the text; adding a reference to Figure 1 in each checklist description would improve clarity.
- [Section 6.1] The statement 'Our evaluation (Sect. 5) verifies the appropriateness of this approach' overstates what a three-person expert interview can verify. I suggest rewording to 'provides initial support for' or 'suggests the appropriateness of'.
- [Table 2] The average ratings for P1 and P2 are both 4.17, but the paper does not report individual-item standard deviations or a measure of agreement; given the small n, reporting the full response distribution (already shown) and avoiding the word 'agree' for the usefulness criterion where Q6=3.66 would be more precise.
Circularity Check
No significant circularity: the guide is a design artifact built on prior survey work with external expert validation, not a derivation that reduces to its own inputs.
full rationale
The paper does not make a predictive or first-principles derivation claim; it presents a reading guide whose structure maps a checklist onto literature categories inherited from the authors' earlier survey [JLK*25]. Although this is a heavy self-citation, the prior survey is an external, checkable artifact (papers can be inspected to verify their assigned classes), and the guide's usefulness is independently assessed by three experts who are explicitly not among the authors. The limitations the paper itself acknowledges—self-reported expertise may be unreliable and recommendations are coarse-grained—are validity and effectiveness concerns, not circular reductions. There is no fitted parameter renamed as a prediction, no equation whose output is identical to its input, and no uniqueness claim imported from the authors' own work. The central claim is that the guide is actionable and useful, which is supported by non-author expert ratings and is not equivalent by construction to the paper's inputs.
Assumptions & free parameters
assumptions (4)
- domain assumption The literature classification proposed in [JLK*25] accurately organizes work on reliable DR visual analytics.
- domain assumption Yes/no answers to the checklist items measure actual DR expertise.
- domain assumption Expert ratings on significance, comprehensiveness, and usefulness predict real value for novices.
- domain assumption The current literature categories will remain valid over time.
Cite this review
Pith. "Pith review of Navigating High-Dimensional Backstage: A Guide for Exploring Literature for the Reliable Use of Dimensionality Reduction." pith.science (2026). https://pith.science/paper/PFOBPCBG
@misc{pith2026250614820,
author = {Pith},
title = {Pith review of: Navigating High-Dimensional Backstage: A Guide for Exploring Literature for the Reliable Use of Dimensionality Reduction},
year = {2026},
howpublished = {\url{https://pith.science/paper/PFOBPCBG}},
note = {Machine review of arXiv:2506.14820}
}
read the original abstract
Visual analytics using dimensionality reduction (DR) can easily be unreliable for various reasons, e.g., inherent distortions in representing the original data. The literature has thus proposed a wide range of methodologies to make DR-based visual analytics reliable. However, the diversity and extensiveness of the literature can leave novice analysts and researchers uncertain about where to begin and proceed. To address this problem, we propose a guide for reading papers for reliable visual analytics with DR. Relying on the previous classification of the relevant literature, our guide helps both practitioners to (1) assess their current DR expertise and (2) identify papers that will further enhance their understanding. Interview studies with three experts in DR and data visualizations validate the significance, comprehensiveness, and usefulness of our guide.
Figures
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry.original add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 i...
-
[3]
: Sanity check for class-coloring-based evaluation of dimension reduction techniques
Aupetit M. : Sanity check for class-coloring-based evaluation of dimension reduction techniques. In Proc. of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization (2014), p. 134–141. https://doi.org/10.1145/2669557.2669578 doi:10.1145/2669557.2669578
arXiv 2014
-
[4]
Cunningham J. P., Ghahramani Z. : Linear dimensionality reduction: Survey, insights, and generalizations. The Journal of Machine Learning Research 16, 1 (2015), 2859--2900. URL: https://jmlr.org/papers/v16/cunningham15a.html
work page 2015
-
[5]
A Critical Analysis of the Usage of Dimensionality Reduction in Four Domains
Cashman D., Keller M., Jeon H., Kwon B. C., Wang Q. : A critical analysis of the usage of dimensionality reduction in four domains. arXiv preprint arXiv:2503.08836 (2025)
work page Pith review arXiv 2025
-
[6]
Choo J., Lee H., Liu Z., Stasko J., Park H. : An interactive visual testbed system for dimension reduction and clustering of large-scale high-dimensional data . In Visualization and Data Analysis 2013 (2013), vol. 8654, SPIE, p. 865402. URL: https://doi.org/10.1117/12.2007316, https://doi.org/10.1117/12.2007316 doi:10.1117/12.2007316
-
[7]
Chatzimparmpas A., Martins R. M., Kerren A. : t- viSNE : Interactive assessment and interpretation of t-SNE projections. IEEE Transactions on Visualization and Computer Graphics 26, 8 (2020), 2696--2714. https://doi.org/10.1109/TVCG.2020.2986996 doi:10.1109/TVCG.2020.2986996
-
[8]
Espadoto M., Martins R. M., Kerren A., Hirata N. S. T., Telea A. C. : Toward a quantitative survey of dimension reduction techniques. IEEE Transactions on Visualization and Computer Graphics 27, 3 (2021), 2153--2173. https://doi.org/10.1109/TVCG.2019.2944182 doi:10.1109/TVCG.2019.2944182
arXiv 2021
Show all 33 references
-
[9]
Etemadpour R., Motta R., Paiva J. G. d. S., Minghim R., de Oliveira M. C. F., Linsen L. : Perception-based evaluation of projection methods for multidimensional data visualization. IEEE Transactions on Visualization and Computer Graphics 21, 1 (2015), 81--94. https://doi.org/1...
2015
-
[10]
G., Fatore F
Fadel S. G., Fatore F. M., Duarte F. S., Paulovich F. V. : Loch: A neighborhood-based multidimensional projection technique for high-dimensional sparse spaces. Neurocomputing 150 (2015), 546--556. https://doi.org/10.1016/j.neucom.2014.07.071 doi:10.1016/j.neucom.2014.07.071
2015 doi
-
[11]
: DimReader : Axis lines that explain non-linear projections
Faust R., Glickenstein D., Scheidegger C. : DimReader : Axis lines that explain non-linear projections. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2019), 481--490. https://doi.org/10.1109/TVCG.2018.2865194 doi:10.1109/TVCG.2018.2865194
2019
-
[12]
E., Salakhutdinov R
Hinton G. E., Salakhutdinov R. R. : Reducing the dimensionality of data with neural networks. Science 313, 5786 (2006), 504--507. https://doi.org/10.1126/science.1127647 doi:10.1126/science.1127647
2006 doi
-
[13]
A., Paulovich F
Joia P., Coimbra D., Cuminato J. A., Paulovich F. V., Nonato L. G. : Local affine multidimensional projection. IEEE Transactions on Visualization and Computer Graphics. 17, 12 (2011), 2563--2571. https://doi.org/10.1109/TVCG.2011.220 doi:10.1109/TVCG.2011.220
2011 doi
-
[14]
: ZADU : A python library for evaluating the reliability of dimensionality reduction embeddings
Jeon H., Cho A., Jang J., Lee S., Hyun J., Ko H.-K., Jo J., Seo J. : ZADU : A python library for evaluating the reliability of dimensionality reduction embeddings. In Proc. IEEE Visualization and Visual Analytics (VIS) (2023), pp. 196--200. https://doi.org/10.1109/VIS54172.202...
2023
-
[15]
: Classes are not clusters: Improving label-based evaluation of dimensionality reduction
Jeon H., Kuo Y.-H., Aupetit M., Ma K.-L., Seo J. : Classes are not clusters: Improving label-based evaluation of dimensionality reduction. IEEE Transactions on Visualization and Computer Graphics 30, 1 (2024), 781--791. https://doi.org/10.1109/TVCG.2023.3327187 doi:10.1109/TVC...
2024
-
[16]
: Measuring and explaining the inter-cluster reliability of multidimensional projections
Jeon H., Ko H.-K., Jo J., Kim Y., Seo J. : Measuring and explaining the inter-cluster reliability of multidimensional projections. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 551--561. https://doi.org/10.1109/TVCG.2021.3114833 doi:10.1109/TVCG.2021.3114833
2021
-
[17]
: Uniform manifold approximation with two-phase optimization
Jeon H., Ko H.-K., Lee S., Jo J., Seo J. : Uniform manifold approximation with two-phase optimization. In Proc. IEEE Visualization and Visual Analytics (VIS) (2022), pp. 80--84. https://doi.org/10.1109/VIS54862.2022.00025 doi:10.1109/VIS54862.2022.00025
2022
-
[18]
: Unveiling high-dimensional backstage: A survey for reliable visual analytics with dimensionality reduction, 2025
Jeon H., Lee H., Kuo Y.-H., Yang T., Archambault D., Ko S., Fujiwara T., Ma K.-L., Seo J. : Unveiling high-dimensional backstage: A survey for reliable visual analytics with dimensionality reduction, 2025. URL: https://arxiv.org/abs/2501.10168, http://arxiv.org/abs/2501.10168 ...
2025 arXiv
-
[19]
: CheckViz : Sanity check and topological clues for linear and non-linear mappings
Lespinats S., Aupetit M. : CheckViz : Sanity check and topological clues for linear and non-linear mappings. Computer Graphics Forum 30, 1 (2011), 113--125. https://doi.org/10.1111/j.1467-8659.2010.01835.x doi:10.1111/j.1467-8659.2010.01835.x
2011
-
[20]
: Topological autoencoders
Moor M., Horn M., Rieck B., Borgwardt K. : Topological autoencoders. In Proceedings of the 37th International Conference on Machine Learning (13--18 Jul 2020), III H. D., Singh A., (Eds.), vol. 119 of Proceedings of Machine Learning Research, PMLR, pp. 7045--7054
2020
-
[21]
G., Aupetit M
Nonato L. G., Aupetit M. : Multidimensional projection for visual analytics: Linking techniques with distortions, tasks, and layout enrichment. IEEE Transactions on Visualization and Computer Graphics 25, 8 (2019), 2650--2673. https://doi.org/10.1109/TVCG.2018.2846735 doi:10.1...
2019
-
[22]
E., Mueller K
Nam J. E., Mueller K. : Tripadvisor ^ N-D : A tourism-inspired high-dimensional space exploration framework with overview and detail. IEEE Transactions on Visualization and Computer Graphics 19, 2 (2013), 291--305. https://doi.org/10.1109/TVCG.2012.65 doi:10.1109/TVCG.2012.65
2013 doi
-
[23]
: Hierarchical stochastic neighbor embedding
Pezzotti N., H \"o llt T., Lelieveldt B., Eisemann E., Vilanova A. : Hierarchical stochastic neighbor embedding. Computer Graphics Forum 35, 3 (2016), 21--30. https://doi.org/https://doi.org/10.1111/cgf.12878 doi:https://doi.org/10.1111/cgf.12878
2016 doi
-
[24]
P., Hanrahan P
Sips M., Neubert B., Lewis J. P., Hanrahan P. : Selecting good views of high-dimensional data using class consistency. Computer Graphics Forum 28, 3 (2009), 831--838. https://doi.org/https://doi.org/10.1111/j.1467-8659.2009.01467.x doi:https://doi.org/10.1111/j.1467-8659.2009.01467.x
2009
-
[25]
: Global versus local methods in nonlinear dimensionality reduction
Silva V., Tenenbaum J. : Global versus local methods in nonlinear dimensionality reduction. In Advances in Neural Information Processing Systems (2002), vol. 15
2002
-
[26]
A., Peltonen J., Weiskopf D., North S
Sacha D., Zhang L., Sedlmair M., Lee J. A., Peltonen J., Weiskopf D., North S. C., Keim D. A. : Visual interaction with dimensionality reduction: A structured literature analysis. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 241--250. https://doi.org/...
2017
-
[27]
: Accelerating t-sne using tree-based algorithms
Van Der Maaten L. : Accelerating t-sne using tree-based algorithms. J. Mach. Learn. Res. 15, 1 (Jan. 2014), 3221–3245
2014
-
[28]
: Visualizing data using t-SNE
van der Maaten L., Hinton G. : Visualizing data using t-SNE . Journal of Machine Learning Research 9, 86 (2008), 2579--2605
2008
-
[29]
O., van den Herik H
Van Der Maaten L., Postma E. O., van den Herik H. J., et al. : Dimensionality reduction: A comparative review. Journal of Machine Learning Research 10, 66-71 (2009), 13
2009
-
[30]
Xia J., Ye F., Chen W., Wang Y., Chen W., Ma Y., Tung A. K. : Ldsscanner: Exploratory analysis of low-dimensional structures in high-dimensional datasets. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 236--245. https://doi.org/10.1109/TVCG.2017.2744098...
2018
-
[31]
: Revisiting dimensionality reduction techniques for visual cluster analysis: An empirical study
Xia J., Zhang Y., Song J., Chen Y., Wang Y., Liu S. : Revisiting dimensionality reduction techniques for visual cluster analysis: An empirical study. IEEE Transactions on Visualization and Computer Graphics (2021), 1--1. https://doi.org/10.1109/TVCG.2021.3114694 doi:10.1109/TV...
2021
-
[32]
S., Melton R., Stasko J., Jacko J
Yi J. S., Melton R., Stasko J., Jacko J. A. : Dust & magnet: Multivariate information visualization using a magnet metaphor. Information Visualization 4, 4 (Oct. 2005), 239–256. https://doi.org/10.1057/palgrave.ivs.9500099 doi:10.1057/palgrave.ivs.9500099
2005 doi
-
[33]
: Interpreting high-dimensional projections with capacity
Zhang Y., Liu J., Lai C., Zhou Y., Chen S. : Interpreting high-dimensional projections with capacity. IEEE Transactions on Visualization and Computer Graphics 30, 9 (2024), 6038--6055. https://doi.org/10.1109/TVCG.2023.3324851 doi:10.1109/TVCG.2023.3324851
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.