{"id":"f4ad82c3-7402-410d-83bd-84a2b489bbde","arxiv_id":"1908.01277","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Treemaps were slower and less accurate for navigation and hierarchy-understanding tasks than icicle plots and the new sundown chart, which performed similarly.","lead":"A comparison of treemaps, icicle plots, sunburst charts, and a new half-circle design for hierarchical data found treemaps slowest and least preferred. Icicle plots and the new design performed about equally.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Accuracy half of the headline is unsupported: only Q3 Friedman p=.0498 with post-hoc p=.0855; treemap speed and preference findings are solid.","rationale":"I read the paper as a comparative usability evaluation with a conditional recommendation: icicle plots and sundown charts are comparable, and treemaps are worst in this setup. The speed and preference findings are strongly supported by the reported statistics, and the authors have provided open-source code and a deployed demo, which is real evidence. The weakest point is not primarily the acknowledged external-validity limitation about randomised data, though that is real; it is that the accuracy claim in the abstract and conclusion is stronger than the paper's own inferential statistics. The only hierarchy-understanding accuracy effect is a borderline omnibus test with non-significant post-hoc p-values, so 'slower performance and accuracy' is not warranted as stated. This matches part of the reader's rationale, where the accuracy overstatement was noted, but the reader's weakest_assumption focused on random data; my concern is internal to the reported analysis. The appropriate disposition remains CONDITIONAL: accept the speed and preference conclusions, but require the accuracy claim to be hedged or removed. Since the reader already recommended a conditional acceptance with revision of the conclusion, my stress-test does not move the verdict.","tokens_in":9112,"tokens_out":3911,"duration_ms":43455,"concrete_test":"Use the open-source logged data from the study repository to re-estimate Q3 and Q4 accuracy with a participant-random-intercept logistic mixed model or an exact paired binary test, and report 95% confidence intervals for treemap-versus-icicle and treemap-versus-sundown accuracy differences. If the contrasts remain non-significant or the intervals include zero, the abstract and conclusion should be revised to say treemap was slower and tended to be less accurate in hierarchy understanding, or that only one of the two hierarchy-understanding tasks showed a marginal accuracy effect.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's advertised central claim is that treemap had 'slower performance and accuracy in hierarchy understanding tasks.' The speed component is supported: Section 4.3 reports significant time effects for Q1, Q3 and Q4, with treemap significantly slower than icicle and sundown charts on Q1 and Q3, and significantly slower than icicle on Q4. The accuracy component is not supported by the reported statistics. For Q3, the only accuracy task with a significant Friedman test, the omnibus result is borderline (chi-square=6, p=.0498), and the post-hoc pairwise comparisons between treemap and either alternative give p=.0855, above the conventional threshold. Q4 accuracy showed no significant effect, and Q1 and Q2 accuracy also showed no significant effect. Therefore the abstract and conclusion overstate the evidence when they say treemap was slower 'and' less accurate. This is load-bearing because the accuracy disadvantage is part of the headline contribution; the speed and preference conclusions can stand alone. The authors' own acknowledged limitation about randomised data is a separate external-validity concern, but the accuracy overstatement is internal to the reported analysis and more directly undermines the stated claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports two user studies comparing treemaps, icicle plots, sunburst charts, and a new semicircular 'sundown' variant for interactive hierarchical quantitative data. After a six-participant pilot in which sunburst was least preferred, a controlled within-subject study with twelve participants compared icicle plot, sundown chart, and treemap on four tasks (navigation, size comparison, relationship identification, least common ancestor), measuring time, accuracy, and preference. The main reported findings are that treemap was significantly slower on Q1, Q3, and Q4, least preferred, and, according to the abstract and conclusion, less accurate on hierarchy-understanding tasks, while icicle and sundown performed similarly. The paper also contributes a consistent overview+detail interaction framework and an open-source WebGL/D3 implementation.","tokens_in":9430,"tokens_out":4326,"duration_ms":47096,"significance":"If the results hold, the paper provides a useful, methodologically more controlled comparison than earlier work, with a consistent interaction design across conditions, a larger hierarchy than most prior studies, and an open-source implementation that supports reproducibility via the linked repository and deployed demo. The speed and preference findings for treemap are clearly supported by the reported statistics. The accuracy claim is not supported at conventional levels, so the headline contribution needs to be narrowed. The sundown chart is a modest design variant rather than a radically new technique, but its inclusion is reasonable for the comparison.","major_comments":[{"comment":"The abstract and conclusion claim that the treemap had 'slower performance and accuracy in hierarchy understanding tasks.' The accuracy component is not supported by the reported statistics. In §4.3, the only accuracy result with a significant omnibus Friedman test is Q3 (chi-square(2)=6, p=.0498), and the post-hoc pairwise comparisons between treemap and either alternative are p=.0855. Q1, Q2, and Q4 accuracy show no significant effect. This is load-bearing because the abstract presents speed and accuracy together as the core result. Please rephrase to state that the treemap was significantly slower and showed a nonsignificant trend toward lower accuracy on Q3, and report the exact p values in the abstract or conclusion.","section":"Abstract; §5 Conclusion"},{"comment":"The study's external validity rests on the randomized quantitative leaf values assigned to the CHI Browse-Off hierarchy. As the authors acknowledge in §5, this may be unrepresentative of real-world datasets. Because Q2 (size comparison) and Q3 (relationship identification) depend on the value distribution, a different distribution (e.g., skewed values or values correlated with hierarchy depth) could change the ordering of visualisations. This is not an internal inconsistency, but it is a correctness-risk concern. The paper should temper the generalised conclusions and specify a concrete replication direction, such as using real value distributions from disk-usage or budget data, rather than presenting the randomized-data version as the primary evidence.","section":"§4.2; §5"}],"minor_comments":[{"comment":"The random quantitative value assignment is not fully specified: the distribution, range, and whether internal node values are computed as sums of children are not stated, which makes the study difficult to reproduce exactly.","section":"§4.2"},{"comment":"The sentence 'Answers were not normally distributed' is imprecise for binary accuracy data; the Friedman test is appropriate for ordinal/binary repeated measures, but the wording suggests a normality assumption that does not apply.","section":"§4.3"},{"comment":"The caption says '95% confidence interval of time' but does not indicate whether the intervals are adjusted for the within-subject design; please clarify the calculation method in the caption or methods.","section":"Figure 2"},{"comment":"The claim that the hybrid WebGL/D3 implementation supports smooth interaction with more than 50,000 nodes is not evaluated in the user study, so it should be presented as a technical benchmark rather than as part of the empirical comparison.","section":"§3 Implementation"},{"comment":"There is a typo in the introduction's summary of results: 'leading to slower performance task accuracy' should read 'leading to slower performance and lower task accuracy'.","section":"§1 Introduction; §5 Conclusion"},{"comment":"The sample size of twelve participants, though common in controlled visualization studies, is small; a sentence acknowledging the limited statistical power for detecting small accuracy differences would help calibrate the conclusions.","section":"§5 Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a visualization journal and is generally well executed. The main issue is the overstatement of the accuracy result; the speed and preference findings are solid and the open-source implementation is a concrete strength. I believe the accuracy overstatement can be fixed with careful rewording and fuller reporting, so I am recommending major revision rather than rejection. I have no concerns about authorship or novelty disclosure; the sundown chart is a minor extension, but the comparative study is useful."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely useful empirical comparison of treemaps, icicle plots, and a new 'sundown' chart for hierarchical quantitative data, with a careful interactive framework and open-sourced implementation. The speed and preference findings are solid. The accuracy half of the headline claim, however, does not survive contact with the paper's own statistics, and that needs to be fixed before the paper can be cited as evidence that treemaps are less accurate.\n\nWhat is actually new: the sundown chart itself is a modest visual variant (a rotated Information Slice, similar to fan charts), but the evaluation is new. Prior comparisons used smaller datasets, limited interaction, and lower-resolution displays. The authors built a consistent overview+detail and zoom-in-place framework across all conditions, with a WebGL/D3 hybrid that handles over 50,000 nodes. That is a real engineering contribution, and they open-sourced the code. The study design is competent: within-subject, counterbalanced presentation, appropriate statistical choices (Friedman for accuracy and preferences, linear mixed models on log-transformed response times).\n\nThe soft spots are proportional. The main one is the accuracy claim. In Section 4.3, the only accuracy task with a significant omnibus result is Q3, and it is borderline (chi-square = 6, p = .0498); the post-hoc pairwise comparisons between treemap and each alternative give p = .0855, above the conventional threshold. Q4 accuracy showed no effect, and neither did Q1 or Q2. So the abstract's 'slower performance and accuracy in hierarchy understanding tasks' is not supported for accuracy. The conclusion repeats the same overstatement. The speed results are unambiguous, and the preference results are solid, so the overall recommendation (icicle and sundown comparable, treemap worst) probably holds, but the accuracy component needs to be toned down. The randomized leaf-value limitation is acknowledged and is a legitimate external-validity concern, not a fatal one. I would also note that dropping the sunburst after the pilot means the comparison is really among three designs, not four as the intro suggests.\n\nThis is a paper for HCI and visualization researchers who design hierarchical quantitative displays, and for tool builders. It deserves serious peer review; with a revision to align the abstract and conclusion with the reported statistics, and ideally posting the raw data, it would be a solid, citable evaluation. I would bring it to our reading group.","headline":"Solid speed and preference results for a useful three-way comparison, but the accuracy claim in the abstract is overstated.","tokens_in":9835,"tokens_out":3009,"would_cite":true,"duration_ms":27859,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treemaps trail icicle and sundown charts in user study","keywords":["hierarchical quantitative data","treemap","icicle plot","sunburst chart","sundown chart","user study","interactive navigation","WebGL visualization"],"falsifier":"Run the same four tasks with the same interaction framework on a real-world hierarchy with a skewed value distribution, such as a disk-usage tree where a few large files dominate, and compare treemap with icicle and sundown charts; if treemap is no longer slower or less accurate, the paper's ordering is specific to the random-value dataset.","tokens_in":8893,"feed_emoji":"📊","tokens_out":6182,"duration_ms":58247,"temperature":0.7,"pith_summary":"This paper asks which visual encoding best supports people who must work interactively with hierarchical quantitative data, meaning trees whose leaves carry numbers, such as disk usage or budget breakdowns. It compares treemaps, icicle plots, and sunburst charts, and introduces a semicircular sunburst variant called the sundown chart. In a controlled study, treemaps were the slowest and least preferred for basic navigation and hierarchy-understanding tasks, while icicle plots and sundown charts performed about equally, with a slight preference for the icicle plot. The paper also reports an interaction framework with overview plus detail and zoom-in-place that keeps visualisations responsive beyond 50,000 nodes, which is what makes the comparison applicable to large datasets.","feed_headline":"Treemaps trail icicle and sundown charts in user study","feed_subtitle":"In a 12-person controlled study, treemaps were slower and less liked for navigating and reading hierarchical numeric data.","key_machinery":"The argument is carried by an interaction framework held constant across visualisations: an overview plus detail view, zoom-in-place navigation, a breadcrumb trail, dynamic labels, a value scale, and unit visualisation, meaning every node is shown rather than aggregated. The sundown chart is the newly proposed object, a semicircular sunburst with the root at the centre and children fanning outward, encoding quantity by arc angle and fitting 16:9 displays better than a full circle. Because the interaction design is the same everywhere, the measured differences are attributed to the visual encodings themselves rather than to navigation features.","core_discovery":"On the paper's own terms, the central discovery is that for interactive hierarchies with quantitative values, treemaps are measurably worse than icicle plots and sundown charts. With 12 participants, the treemap was significantly slower on a basic navigation task, averaging 53.0 seconds versus 32.6 seconds for the icicle plot and 35.2 seconds for the sundown chart; it was also significantly slower on a closest-relationship task, averaging 44.2 seconds versus 20.0 and 20.6 seconds, slower on a least-common-ancestor task, and less accurate on the relationship task, 91.4 percent versus 97.2 percent for both alternatives. Users ranked the treemap lowest for aesthetics, usability, and overall preference. The paper finds no support for the suspected trade-off between hierarchy-friendly and quantity-friendly designs: icicle plots and sundown charts handle both kinds of tasks well.","pith_inferences":["We extrapolate that the treemap's disadvantage may depend on the squarified layout chosen here; a different treemap layout with less extreme aspect ratios could narrow the gap, since the paper does not vary layout strategies within visualisation type.","The motion-sickness reports for the sundown chart suggest a testable follow-up: measuring disorientation over longer sessions could reveal whether its semicircular geometry trades comfort for screen fit.","Because the leaf values were assigned randomly, we infer that the ordering is most secure for hierarchies with roughly uniform leaf values; a natural extension is to re-run the same four tasks on real-world distributions such as disk usage, where a few large leaves dominate.","The similar performance of the two one-dimensional encodings predicts that other length-based hierarchy views, such as flame graphs, will also beat treemaps on navigation and hierarchy-understanding tasks."],"forward_implications":["Designers of tools for hierarchical quantitative data have a reason to offer icicle plots or sundown charts alongside, or instead of, treemaps when users must navigate and understand tree structure.","The sundown chart provides a radial encoding competitive with the linear icicle plot while avoiding the large white-space cost of a full 360-degree sunburst on wide screens.","The absence of a hierarchy-versus-quantity trade-off suggests that one-dimensional encodings such as length or angle can support both structural and value-comparison tasks at once.","The WebGL-and-D3 hybrid implementation shows that these non-treemap encodings remain responsive for hierarchies with more than 50,000 nodes, removing a scale barrier that earlier evaluations had."],"supporting_citations":[{"why":"Introduces the treemap, the incumbent design whose performance is the paper's main negative result.","marker":"[23]"},{"why":"Introduces the icicle plot as a hierarchy display, one of the two top-performing designs.","marker":"[27]"},{"why":"Introduces the sunburst chart and the focus+context radial navigation ideas that the sundown chart adapts.","marker":"[40]"},{"why":"Information Slices, the cascading semicircular-disc design on which the sundown chart is based.","marker":"[3]"},{"why":"Unit visualisations, the basis for showing all nodes without aggregation during zooming.","marker":"[14]"},{"why":"Flame Graph, which supplies the zoom-in-place interaction pattern used in the framework.","marker":"[20]"},{"why":"SyncTRACE, which provides the two-level overview concept used in the common navigation design.","marker":"[24]"},{"why":"Perceptual guidelines that justify the squarified treemap layout and the choice of aspect-ratio controls.","marker":"[26]"},{"why":"Earlier comparison of treemap, icicle plot and sunburst that the study updates with larger data and richer interaction.","marker":"[4]"},{"why":"Recent usability study of icicle plots and sunburst charts that motivates the need for hierarchy understanding with quantitative attributes.","marker":"[18]"}],"fun_headline_variants":["Treemaps lose to icicle and sundown charts","Icicle and sundown charts outpace treemaps","Treemaps slowest and least liked in study","Sundown chart matches icicle, beats treemap"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result rests on the assumption that a single hierarchy with randomly assigned leaf values behaves like real-world hierarchical quantitative data; the paper itself notes that real datasets may have different shapes and value distributions.","fun_headline_variants_meta":{"raw":{"variants":["Treemaps lose to icicle and sundown charts","Icicle and sundown charts outpace treemaps","Treemaps slowest and least liked in study","Sundown chart matches icicle, beats treemap"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1320,"prompt_tokens":835,"completion_tokens":485,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":451,"completion_tokens_details":{"reasoning_tokens":418}},"tokens_in":451,"tokens_out":485,"duration_ms":5238,"temperature":1.0,"reasoning_tokens":418,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:16:27.168833+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same four tasks with the same interaction framework on a real-world hierarchy with a skewed value distribution, such as a disk-usage tree where a few large files dominate, and compare treemap with icicle and sundown charts; if treemap is no longer slower or less accurate, the paper's ordering is specific to the random-value dataset.","supporting_citations":[{"cited_title":"Johnson and B","cited_arxiv_id":null,"evidence_quote":"Introduces the treemap, the incumbent design whose performance is the paper's main negative result."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the icicle plot as a hierarchy display, one of the two top-performing designs."},{"cited_title":"Stasko and E","cited_arxiv_id":null,"evidence_quote":"Introduces the sunburst chart and the focus+context radial navigation ideas that the sundown chart adapts."},{"cited_title":"Andrews and H","cited_arxiv_id":null,"evidence_quote":"Information Slices, the cascading semicircular-disc design on which the sundown chart is based."},{"cited_title":"Drucker and R","cited_arxiv_id":null,"evidence_quote":"Unit visualisations, the basis for showing all nodes without aggregation during zooming."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Flame Graph, which supplies the zoom-in-place interaction pattern used in the framework."},{"cited_title":"Karran, J","cited_arxiv_id":null,"evidence_quote":"SyncTRACE, which provides the two-level overview concept used in the common navigation design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Perceptual guidelines that justify the squarified treemap layout and the choice of aspect-ratio controls."},{"cited_title":"Barlow and P","cited_arxiv_id":null,"evidence_quote":"Earlier comparison of treemap, icicle plot and sunburst that the study updates with larger data and richer interaction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Recent usability study of icicle plots and sunburst charts that motivates the need for hierarchy understanding with quantitative attributes."}],"review_version":1}