REVIEW 3 major objections 4 minor 40 references
Evaluating Ordering Strategies of Star Glyph Axes
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper reports a user study showing that dissimilarity-based star-glyph ordering improves clustering accuracy over similarity-based ordering, with the largest advantage when the data includes clutter dimensions.
desk verdict Useful first comparison of star-glyph axis orderings, but the central DIS advantage rests on non-independent trial-level statistics. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the eight-axis star glyph, whose axes can be permuted. The paper defines two orderings: similarity-based (SIM), computed by exhaustive search for the permutation maximizing pairwise Euclidean similarity between adjacent dimensions, yielding homogeneous convex shapes; and dissimilarity-based (DIS), the same search minimizing adjacency similarity, yielding spiky salient shapes. The benchmark uses synthetic eight-dimensional datasets with two to four known clusters plus manually planted noise and clutter dimensions, validated by a density-based clustering check. User grouping quality is scored by a two-step Jaccard-index procedure that accounts for both user-selected groups and ground-truth clusters.
What would settle it
A permutation test that reassigns the six two-clutter datasets between SIM and DIS and recomputes the median cluster-quality gap would settle whether the 0.58-versus-0.52 advantage is a property of the ordering strategy or of particular datasets; if the gap does not survive random reassignments, the claimed DIS advantage under clutter would not hold.
Extended reading notes
Core claim
The paper's central claim is that dissimilarity-based star-glyph layouts help users in visual clustering tasks more than similarity-based layouts, and that this advantage grows when the data contains clutter dimensions. Participants were significantly more accurate under the dissimilarity ordering than the similarity ordering overall, with median cluster quality 0.69 versus 0.67 (p < .05), and the gap was largest with two clutter dimensions: 0.58 versus 0.52 (p < .001). Noise identification also favored the dissimilarity ordering at that clutter level, while completion time and confidence showed no significant overall difference. These results are presented as direct support for the hypothesis that salient, spiky shapes aid grouping.
Load-bearing premise
The load-bearing premise is that the handful of datasets within each clutter level are interchangeable between the SIM and DIS conditions, so the measured DIS advantage is not an artifact of easier datasets having been assigned to DIS.
Editorial extensions
If this is right
- For datasets with clutter, glyph designers should prefer dissimilarity-based axis ordering to support clustering and noise identification.
- For clutter-free datasets, similarity-based ordering is at least as accurate and trends faster and more confident, so it remains a reasonable default there.
- Adding clutter dimensions substantially degrades cluster quality, noise identification, and user confidence, so suppressing or removing clutter dimensions before glyph display is advisable.
- Users reported seeing clusters more clearly with spiky dissimilarity-ordered glyphs, matching the measured accuracy advantage even though confidence ratings did not differ significantly.
Reading between the lines
- If the advantage comes from spike saliency, a natural next test is whether a few dominant spikes suffice or whether the full dissimilarity ordering is needed; the paper leaves the required number of spikes open.
- The results suggest an adaptive rule: detect clutter dimensions automatically and switch from similarity to dissimilarity ordering only when clutter is present; this rule is testable against either fixed strategy.
- The same ordering logic could be examined in other axes-based radial visualizations, such as RadViz or parallel coordinates, since their axis-ordering effects are task-dependent; the paper does not make this claim.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a controlled user study (N = 15) comparing two star-glyph axis ordering strategies, similarity-based (SIM) and dissimilarity-based (DIS), for a visual grouping/noise-identification task. Eighteen synthetic 50-point datasets with 0, 1, or 2 clutter dimensions serve as ground truth, and each participant performs 18 trials (all datasets, with 3 SIM and 3 DIS trials per clutter level). The main claims are that (H1) clutter degrades cluster quality, noise identification, and confidence, and (H2) DIS ordering yields better cluster quality than SIM, especially under clutter, and better noise identification in the 2-clutter-dimension condition. The results section reports significant clutter effects on cluster quality and noise identification, a small but significant overall DIS advantage in cluster quality (median 0.69 vs 0.67, p < .05), and a larger DIS advantage in the 2C condition (0.58 vs 0.52, p < .001). Time and confidence effects are largely non-significant. The authors provide public study materials and analysis scripts.
Significance. If the central claim is statistically sound, this is a useful empirical contribution to a sparse literature: previous work (Klippel et al.) suggested that salient/spikey glyph shapes help grouping, but direct comparisons of data-driven ordering strategies with controlled ground truth were missing. The paper also provides practical design considerations and makes the study material, data, and scripts publicly available, which supports reproducibility. The strength of the contribution, however, rests almost entirely on the statistical analysis of the cluster-quality and noise-identification measures; the manuscript currently does not establish that the reported significance survives the repeated-measures structure of the experiment or the small number of datasets per condition.
major comments (3)
- [Section 3.4 (H1b, H2b)] The statistical tests appear to treat the 270 individual trials as independent observations, as indicated by the reported chi-square statistics with N = 270 and by the Wilcoxon/t-tests comparing SIM and DIS. In this design, each of the 15 participants contributes 18 trials, and each of the 18 datasets is seen by every participant, so trial-level observations are not independent. This inflates the effective sample size and may invalidate the reported p-values for the central DIS-over-SIM cluster-quality claim. The authors should state the pairing unit explicitly and re-analyze the data at the participant level (or with a mixed model including random intercepts for participant and dataset), reporting the effect sizes and confidence intervals for the overall comparison and for the 2C condition.
- [Section 3.3 (Post-Processing and Analysis)] The post-processing rule that converts groups containing one or two glyphs into noise is a hand-set threshold, and the manuscript provides no analysis of how this rule affects the SIM versus DIS comparison. Because the two-step Jaccard quality measure is sensitive to exact cluster membership, this rule can change which trials are scored as near-correct, and its effect may differ between orderings. The authors should justify the threshold a priori or include a sensitivity analysis showing that the reported DIS advantage is robust to alternative thresholds (e.g., only singletons, or groups of size up to three).
- [Section 3.2 (Benchmark Datasets)] The DBSCAN verification of the manually created PCDC clusters is reported without any numerical agreement measure. Since the DBSCAN parameters (minPts = 3, epsilon = 0.5) are hand-set, the reader cannot assess whether the ground-truth labels for all 18 datasets are consistent with the intended cluster structure. The authors should report an agreement statistic (e.g., adjusted Rand index or cluster purity) for each dataset, and state whether any dataset failed the verification and how such a failure would be handled.
minor comments (4)
- [Section 3.4 (H2b)] Multiple SIM-versus-DIS comparisons are reported (overall and within 0C, 1C, 2C) without an adjustment for multiple testing; the authors should state whether the reported p-values are corrected or interpret the uncorrected comparisons as exploratory.
- [Figure 3] Figure 3 shows the cluster-quality distributions but does not include effect sizes or confidence intervals, and the axes/legend are not described in the caption; adding these would help readers judge the practical size of the DIS advantage.
- [Section 3.3 (Randomization)] The trials are blocked by increasing difficulty (0C, then 1C, then 2C) for all participants, which may introduce order or fatigue effects that are confounded with clutter level; this design choice should be discussed as a limitation or mitigated in the analysis.
- [General] The manuscript contains minor typographical issues (e.g., 'execute a more coherent analysis', 'influenced', and inconsistent use of 'ε' vs 'eps' for the DBSCAN parameter); these do not affect the content but should be cleaned up.
Circularity Check
No circularity: the central claim is an empirical comparison against externally defined synthetic ground truth, not a derivation from fitted inputs or author-authored benchmarks.
full rationale
This paper is an empirical user study, not a derivation chain: it compares two star glyph axis orderings (SIM and DIS) on manually created PCDC datasets whose cluster structure is checked with DBSCAN, and it measures task completion time, cluster quality, noise identification quality, and confidence. There is no fitted parameter that is later renamed as a prediction, and no equation whose output equals its input by construction. The DIS-over-SIM cluster quality advantage reported in Section 3.4 is an observed behavioral difference, not an analytic consequence of how DIS or SIM is defined. The authors cite Klippel et al. for the motivating hypothesis that salient shapes support grouping, but the cited work is external prior research, not a self-citation, and the present study independently tests the hypothesis with new data. The authors' own prior work (e.g., Fuchs et al.) is cited only in peripheral contexts such as contour perception and quality metrics, not as the load-bearing justification for the reported ordering effect. The stated limitations about small trial numbers and possible outlier distortion concern statistical power and generalizability, not circularity. Therefore, the paper is self-contained as an empirical evaluation and exhibits no significant circularity.
Assumptions & free parameters
free parameters (2)
- DBSCAN parameters for ground-truth verification =
minPts=3, eps=0.5
- Small-group-to-noise threshold =
groups with 1 or 2 glyphs
assumptions (5)
- domain assumption Euclidean distance between dimensions adequately captures perceptual (dis)similarity for generating SIM and DIS orderings via exhaustive search.
- domain assumption The manually created PCDC datasets, verified with DBSCAN (minPts=3, eps=0.5), provide valid cluster and noise ground truth.
- ad hoc to paper Post-processing groups with one or two glyphs as noise yields a coherent analysis rather than distorting the SIM-DIS comparison.
- domain assumption Results from star glyphs with eight dimensions, colored axes, and 15 local students generalize to other glyph configurations and user populations.
- domain assumption The random assignment of datasets per clutter level to SIM or DIS balances dataset difficulty so that strategy comparisons are not confounded.
Cite this review
Pith. "Pith review of Evaluating Ordering Strategies of Star Glyph Axes." pith.science (2026). https://pith.science/paper/BXCUWHVT
@misc{pith2026190800576,
author = {Pith},
title = {Pith review of: Evaluating Ordering Strategies of Star Glyph Axes},
year = {2026},
howpublished = {\url{https://pith.science/paper/BXCUWHVT}},
note = {Machine review of arXiv:1908.00576}
}
read the original abstract
Star glyphs are a well-researched visualization technique to represent multi-dimensional data. They are often used in small multiple settings for a visual comparison of many data points. However, their overall visual appearance is strongly influenced by the ordering of dimensions. To this end, two orthogonal categories of layout strategies are proposed in the literature: order dimensions by similarity to get homogeneously shaped glyphs vs. order by dissimilarity to emphasize spikes and salient shapes. While there is evidence that salient shapes support clustering tasks, evaluation, and direct comparison of data-driven ordering strategies has not received much research attention. We contribute an empirical user study to evaluate the efficiency, effectiveness, and user confidence in visual clustering tasks using star glyphs. In comparison to similarity-based ordering, our results indicate that dissimilarity-based star glyph layouts support users better in clustering tasks, especially when clutter is present.
Figures
Reference graph
Works this paper leans on
-
[1]
G. Albuquerque, M. Eisemann, D. J. Lehmann, H. Theisel, and M. A. Magnor. Improving the Visual Analysis of High-dimensional Datasets Using Quality Measures. In Proc. of the IEEE Conf. on Visual Analytics Science and Technology, pp. 19–26. IEEE Computer Society, 2010. doi: 10.1109/V AST.2010.5652433
arXiv 2010
-
[2]
N. V . Andrienko and G. L. Andrienko.Exploratory Analysis of Spatial and Temporal Data: A Systematic Approach. Springer, 2006. doi: 10. 1007/3-540-31190-4
work page 2006
-
[3]
M. Ankerst, S. Berchtold, and D. A. Keim. Similarity Clustering of Dimensions for an Enhanced Visualization of Multidimensional Data. In IEEE Symp. on Information Visualization, pp. 52–60, 1998. doi: 10. 1109/INFVIS.1998.729559
-
[4]
A. O. Artero, M. C. F. de Oliveira, and H. Levkowitz. Enhanced High Dimensional Data Visualization through Dimension Reduction and Attribute Arrangement. In Intl. Conf. on Information Visualisation, pp. 707–712, 2006. doi: 10.1109/IV.2006.49
-
[5]
M. Behrisch, M. Blumenschein, N. W. Kim, L. Shao, M. El-Assady, J. Fuchs, D. Seebacher, A. Diehl, U. Brandes, H. Pfister, T. Schreck, D. Weiskopf, and D. A. Keim. Quality metrics for information visual- ization. Comput. Graph. Forum, 37(3):625–662, 2018. doi: 10.1111/ cgf.13446
work page 2018
-
[6]
I. Borg and T. Staufenbiel. Performance of Snow Flakes, Suns, and Factorial Suns in the Graphical Representation of Multivariate Data. Multivariate Behavioral Research, 27(1):43–55, 1992. doi: 10.1207/ s15327906mbr2701 4
work page 1992
-
[7]
R. Borgo, J. Kehrer, D. H. S. Chung, E. Maguire, R. S. Laramee, H. Hauser, M. Ward, and M. Chen. Glyph-based Visualization: Foun- dations, Design Guidelines, Techniques and Applications. In Euro- graphics - State of the Art Reports , pp. 39–63, 2013. doi: 10.2312/ conf/EG2013/stars/039-063
work page 2013
-
[8]
S. Bremm, M. Hess, T. von Landesberger, and D. W. Fellner. PCDC - On the Highway to Data - A Tool for the Fast Generation of Large Syn- thetic Data Sets. In EuroVis Workshop on Visual Analytics. Eurograph- ics Association, 2012. doi: 10.2312/PE/EuroV AST/EuroV A12/007-011
Show all 40 references
-
[9]
L. D. Caro, V . Fr´ıas-Mart´ınez, and E. Fr´ıas-Mart´ınez. Analyzing the Role of Dimension Arrangement for Data Visualization in Radviz. In Advances in Knowledge Discovery and Data Mining , pp. 125–132. Springer, 2010. doi: 10.1007/978-3-642-13672-6 13
2010 doi
-
[10]
Cheng, W
S. Cheng, W. Xu, and K. Mueller. RadViz Deluxe: An Attribute-Aware Display for Multivariate Data. Processes, 5(4):75, 2017. doi: 10.3390/ pr5040075
2017
-
[11]
Dasgupta and R
A. Dasgupta and R. Kosara. Pargnostics: Screen-Space Metrics for Par- allel Coordinates. IEEE Trans. on Vis. and Comp. Graph., 16(6):1017– 1026, 2010. doi: 10.1109/TVCG.2010.184
2010 doi
-
[12]
R. F. Erbacher. Glyph-based generic network visualization. In Visual- ization and Data Analysis, pp. 228–237, 2002. doi: 10.1117/12.458790
2002 doi
-
[13]
Ester, H
M. Ester, H. Kriegel, J. Sander, and X. Xu. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In Proc. of the Intl. Conf. on Knowledge Discovery and Data Mining, pp. 226–231, 1996
1996
-
[14]
Fischer, J
F. Fischer, J. Fuchs, and F. Mansmann. ClockMap: Enhancing Circular Treemaps with Temporal Glyphs for Time-Series Data. InEurographics Conference on Visualization, EuroVis 2012, Vienna, Austria, June 5-8.,
2012
-
[15]
Friendly and E
M. Friendly and E. Kwan. Effect ordering for data displays. Com- putational Statistics & Data Analysis, 43(4):509–539, 2003. doi: 10. 1016/S0167-9473(02)00290-6
2003
-
[17]
Fuchs, P
J. Fuchs, P. Isenberg, A. Bezerianos, and D. A. Keim. A Systematic Review of Experimental Studies on Data Glyphs. IEEE Trans. on Vis. and Comp. Graph., 23(7):1863–1879, 2017. doi: 10.1109/TVCG.2016 .2549018
2017 doi
-
[18]
Garc´ıa, J
S. Garc´ıa, J. Luengo, and F. Herrera. Data Preprocessing in Data Mining, vol. 72 of Intelligent Systems Reference Library . Springer,
-
[19]
Harrower and C
M. Harrower and C. A. Brewer. ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps. The Cartographic Journal, 40(1):27–37, 2003
2003
-
[20]
Hoffman, G
P. Hoffman, G. G. Grinstein, K. A. Marx, I. Grosse, and E. Stanley. DNA visual and analytic data mining. In IEEE Visualization Proc., pp. 437–442, 1997. doi: 10.1109/VISUAL.1997.663916
1997
-
[21]
Inselberg and B
A. Inselberg and B. Dimsdale. Parallel Coordinates: A Tool for Visual- izing Multi-dimensional Geometry. In Proc. IEEE Visualization, pp. 361–378, 1990. doi: 10.1109/VISUAL.1990.146402
1990
-
[22]
Kayaert and J
G. Kayaert and J. Wagemans. Delayed shape matching benefits from simplicity and symmetry. Vision Research, 49(7):708 – 717, 2009. doi: 10.1016/j.visres.2009.01.002
2009 doi
-
[23]
Kintzel, J
C. Kintzel, J. Fuchs, and F. Mansmann. Monitoring large IP spaces with ClockView. In Intl. Symp. on Visualization for Cyber Security, p. 2, 2011. doi: 10.1145/2016904.2016906
2011
-
[24]
Klippel, F
A. Klippel, F. Hardisty, R. Li, and C. Weaver. Colour-Enhanced Star Plot Glyphs: Can Salient Shape Characteristics Be Overcome? Cartographica, 44(3):217–231, 2009. doi: 10.3138/carto.44.3.217
2009 doi
-
[25]
Klippel, F
A. Klippel, F. Hardisty, and C. Weaver. Star Plots: How Shape Characteristics Influence Classification Tasks. Cartography and Ge- ographic Information Science, 36(2):149–163, 2009. doi: 10.1559/ 152304009788188808
2009
-
[26]
Kriegel, P
H. Kriegel, P. Kr¨oger, and A. Zimek. Clustering High-Dimensional Data: A Survey on Subspace Clustering, Pattern-Based Clustering, and Correlation Clustering. ACM Transactions on Knowledge Discovery from Data, 3(1):1:1–1:58, 2009. doi: 10.1145/1497577.1497578
2009
-
[27]
T. V . Long. ArcViz: An Extended Radial Visualization for Classes Separation of High Dimensional Data. In Intl. Conf. on Knowledge and Systems Engineering, pp. 158–162, 2018. doi: 10.1109/KSE.2018. 8573428
2018 doi
-
[28]
Opach, S
T. Opach, S. Popelka, J. Dolezalova, and J. K. Rød. Star and polyline glyphs in a grid plot and on a map display: which perform better? Cartography and Geographic Information Science , 45(5):400–419,
-
[29]
W. Peng, M. O. Ward, and E. A. Rundensteiner. Clutter Reduction in Multi-Dimensional Data Visualization Using Dimension Reordering. In IEEE Symp. on Information Visualization, pp. 89–96, 2004. doi: 10. 1109/INFVIS.2004.15
2004
-
[30]
Ropinski and B
T. Ropinski and B. Preim. Taxonomy and usage guidelines for glyph- based medical visualization. In Simulation and Visualization, pp. 121– 138, 2008
2008
-
[31]
Rze´zniczak
T. Rze´zniczak. Evaluation of multidimensional visualization techniques for medical patterns representation. Journal of Theoretical and Applied Computer Science, 7(4):70–85, 2013
2013
-
[32]
Sacha, A
D. Sacha, A. Stoffel, F. Stoffel, B. C. Kwon, G. P. Ellis, and D. A. Keim. Knowledge Generation Model for Visual Analytics. IEEE Trans. on Vis. and Comp. Graph., 20(12):1604–1613, 2014. doi: 10.1109/TVCG. 2014.2346481
2014
-
[33]
Siegel, E
J. Siegel, E. Farrell, R. Goldwyn, and H. Friedman. The Surgical Implications of Physiologic Patterns in Myocardial Infarction Shock. Surgery, 72(1):126–141, 1972
1972
-
[34]
A. Tatu, G. Albuquerque, M. Eisemann, P. Bak, H. Theisel, M. A. Magnor, and D. A. Keim. Automated Analytical Methods to Support Visual Exploration of High-Dimensional Data. IEEE Trans. on Vis. and Comp. Graph., 17(5):584–597, 2011. doi: 10.1109/TVCG.2010.242
2011 doi
-
[35]
M. O. Ward. Multivariate Data Glyphs: Principles and Practice. In Handbook of Data Visualization, pp. 179–198. Springer, 2008. doi: 10. 1007/978-3-540-33037-0 8
2008
-
[36]
C. Ware. Information Visualization: Perception for Design. San Francisco, CA: Morgan Kaufmann, 2004
2004
-
[37]
J. Yang, W. Peng, M. O. Ward, and E. A. Rundensteiner. Interactive Hi- erarchical Dimension Ordering, Spacing and Filtering for Exploration of High Dimensional Datasets. In IEEE Symp. on Information Visual- ization, pp. 105–112, 2003. doi: 10.1109/INFVIS.2003.1249015
2003 arXiv
-
[38]
Zhang, K
Z. Zhang, K. T. McDonnell, and K. Mueller. A Network-Based Inter- face for the Exploration of High-Dimensional Data Spaces. In IEEE Pacific Visualization Symp., pp. 17–24. IEEE Computer Society, 2012. doi: 10.1109/PacificVis.2012.6183569
2012
-
[2012]
doi: 10.2312/PE/EuroVisShort/EuroVisShort2012/097-101
-
[2015]
doi: 10.1007/978-3-319-10247-4
-
[2018]
doi: 10.1080/15230406.2017.1364169
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.