{"id":"6c8c4a97-e8a6-433f-83b1-3103f2f47347","arxiv_id":"1908.07363","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A benchmark of eight node overlap removal algorithms finds VPSC and R Wordle-L best preserve layout quality, with VPSC the faster of the two.","lead":"This paper compares eight algorithms that remove overlapping nodes in graph drawings, testing them on 854 synthetic and real-world graphs using five quality metrics and running time. It finds that VPSC and R Wordle-L produce the best quality layouts, with VPSC much faster.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The top-two conclusion (VPSC, R Wordle-L) rests on one hand-picked metric per criterion, with no sensitivity analysis or significance testing; if unselected metrics rank algorithms differently, the headline result could change.","rationale":"The reader's weakest assumption is that the five selected metrics faithfully represent the five quality criteria. My stress-test read identifies the same load-bearing point and sharpens it: the paper does not demonstrate that the ranking of algorithms is stable under the choice of representative metric, nor does it provide any statistical test that the observed differences between VPSC/R Wordle-L and their closest competitors are meaningful. This is a correctness-risk issue for the central conclusion, not merely a presentation issue. If the non-selected metrics were computed and the rankings changed, the conclusion 'VPSC and R Wordle-L obtained the best quality results' would not stand as stated. The paper otherwise gives a careful, well-structured protocol, including a large corpus and clearly defined metrics, so the concern does not warrant rejection; it warrants keeping the reader's conditional verdict and adding the robustness analysis as a condition. The proposed concrete check is feasible because the paper already computed correlations among metrics in the same class, implying the underlying embeddings are available for re-scoring.","tokens_in":8416,"tokens_out":4480,"duration_ms":44373,"concrete_test":"Run the full comparison using every non-selected metric in each class from Table 1 (e.g., gs_ch_sd, sp_bb_na, nm_dm_me, el_r, oo_kt) on the same 6,832 embeddings, and compute per-metric rankings of the eight algorithms. Then perform paired Wilcoxon signed-rank tests across the 840 synthetic graphs for VPSC vs. FTA, VPSC vs. PRISM, and R Wordle-L vs. each competitor, on all five selected metrics. If any non-selected metric changes the top-two set, or if the paired differences are not significant at a reasonable threshold, the Sec. 5 conclusion is not robust to the metric-selection decision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Sec. 5 is that 'VPSC and R Wordle-L obtained the best quality results.' The evidence for this is the boxplots in Figs. 1-2, where each of the five quality criteria is represented by exactly one metric. Sections 3.2-3.5 justify the representative metrics by interpretability and pairwise correlation with alternatives (e.g., gs_bb_iar vs. gs_ch_sd, r=0.77), not by showing that the relative ranking of the eight algorithms is insensitive to that choice. A correlation of 0.77 leaves room for rank changes among VPSC, FTA, PRISM, and R Wordle-L. The paper reports no paired significance tests or confidence intervals, so the phrase 'best quality results' may reflect differences that are within noise. The concern is not that the experiments are fraudulent or even wrong, but that the headline conclusion is underdetermined by the reported evidence: the chosen representative metrics could be the load-bearing assumption that makes VPSC and R Wordle-L appear superior.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a comparative study of node overlap removal algorithms. The authors first review 21 quality metrics from the literature, classify them into five quality criteria (orthogonal ordering preservation, spread minimization, global shape preservation, node movement minimization, and edge length preservation), and select one representative metric per class, including a new normalized aspect-ratio metric (gs_bb_iar). They then compare eight overlap removal algorithms (Scaling, PFS, PFS', FTA, VPSC, PRISM, R Wordle-L, and GTREE) on 840 synthetic graphs and 14 real-world graphs, using the five selected metrics plus running time. The central conclusion is that VPSC and R Wordle-L achieve the best quality results, with R Wordle-L superior in global shape preservation and comparable on other criteria, while VPSC is faster.","tokens_in":8738,"tokens_out":3314,"duration_ms":34080,"significance":"If the comparison is sound, the paper provides a useful service to visualization designers by organizing a scattered metric literature, proposing a normalized improvement over an existing metric, and offering the first broad empirical comparison of eight overlap removal algorithms on a substantial benchmark. The experimental protocol is clearly described, the dataset is large and diverse, and the authors are careful to separate quality dimensions. However, the significance is limited by the subjective selection of representative metrics, the absence of statistical tests, and an unspecified node-size model, all of which bear directly on whether the headline ranking is robust.","major_comments":[{"comment":"The selection of one representative metric per quality criterion is load-bearing for the paper's conclusion, but the authors do not show that the ranking of the eight algorithms is insensitive to this choice. For example, gs_bb_iar is preferred over gs_ch_sd based on a correlation coefficient of 0.77; a correlation that leaves 40% unexplained can still permit rank changes among the top algorithms such as VPSC, FTA, PRISM, and R Wordle-L. I recommend adding a sensitivity analysis that recomputes the rankings under alternative representatives within each class (or across all 21 metrics), so that the reader can see whether the conclusion 'VPSC and R Wordle-L obtained the best quality results' is an artifact of metric choice.","section":"Sec. 3, especially Secs. 3.2–3.5"},{"comment":"The qualitative claims, such as 'R Wordle-L outperforms VPSC on global shape preservation and is comparable on the other criteria,' are based on aggregated boxplots and means without any paired significance tests, confidence intervals, or effect-size measures. The distributions in Figs. 1 and 2 visibly overlap, so differences between the top algorithms may be within noise. I ask the authors to add paired statistical tests (e.g., Wilcoxon signed-rank with appropriate multiple-comparison correction) or bootstrap confidence intervals for the five metrics, and to report effect sizes. This is necessary to support the central ranking claim.","section":"Sec. 4.1, Figs. 1–2"},{"comment":"The experimental protocol never specifies how node widths and heights (w_v, h_v) are assigned for the synthetic and real-world graphs. Overlap removal algorithms and all five quality metrics depend directly on these sizes; without a precise node-size model the experiment is not reproducible, and the results may not generalize to other node-size distributions. The paper should state the node-size generation procedure (e.g., uniform random, proportional to degree or area of the initial FM^3 layout) and ideally include a robustness check across different size models.","section":"Sec. 4, Step 1 (Datasets)"},{"comment":"The sentence 'we did not observe any significant differences in terms of results' across graph types is an overstatement because no statistical tests are reported for this comparison either. This should either be removed or supported by a proper interaction analysis of graph type with algorithm performance.","section":"Sec. 5, Conclusion"}],"minor_comments":[{"comment":"The notation in the column for the normalizing function is ambiguous: '1/k√(2×n)' should be typeset as 1/(k*sqrt(2*n)) or similar, and the variable k should be defined earlier in the same paragraph to avoid confusion with the KNN parameter in nm_knn.","section":"Sec. 3.4, Table 2"},{"comment":"In the formula for el_rsdd, the subscript 'E2_dt' appears twice where the text defines the Delaunay edge set as 'Edt'; please correct these typos.","section":"Sec. 3.5"},{"comment":"The phrase 'as it is proved that they maintain the original orthogonal ordering' would benefit from a citation or a short proof sketch, since the reader cannot verify this claim from the text alone.","section":"Sec. 4.1, first paragraph"},{"comment":"The paper does not state whether the same graph layouts (FM^3 for synthetic, SFDP for real-world) are used as initial embeddings for all algorithms, nor how ties and degenerate cases (e.g., nodes with identical initial coordinates) are handled. A brief clarification would improve reproducibility.","section":"Sec. 4, Step 3"},{"comment":"The phrase 'Scaling optimises 4 out of 5 criteria' is slightly misleading because Scaling's best score on oo_nni is trivially expected from its definition; the reader should be told that the statement refers to the measured medians on the presented datasets.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper is a useful comparative study and the authors have assembled a valuable benchmark, but the central ranking is currently underdetermined by three avoidable gaps: metric-selection sensitivity, lack of statistical tests, and an unspecified node-size model. These are fixable within the manuscript's scope, and I would encourage the authors to address them. I would also note that no data or code availability statement is given, which is increasingly expected for empirical papers of this type."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is the first to compare eight node overlap removal algorithms on a common corpus of 854 graphs, and it organizes 21 metrics into five quality classes. That alone makes it useful for visualization designers who need to pick an algorithm.\n\nThe classification is a real contribution, and the authors make sensible choices in places, like replacing the unbounded aspect-ratio metric with their refined gs_bb_iar. They also report running times, which matters for practical use, and the qualitative rankings in the text are consistent with the boxplots they show.\n\nThe soft spot is the one the stress-test flags: each of the five criteria is represented by exactly one metric, selected by the authors' judgment. The selection is not the problem by itself, but they never show that the relative ranking of the eight algorithms is insensitive to that choice. They report a correlation of 0.77 between gs_bb_iar and gs_ch_sd, which is high but not high enough to rule out rank changes among VPSC, FTA, PRISM, and R Wordle-L. And there is no significance testing or confidence intervals on the boxplots, so 'best quality results' may claim more than the data support. The lack of released code and data also makes it hard to check whether node sizes or dataset construction affect the outcome.\n\nThese are fixable. A sensitivity analysis that reruns the ranking with alternative metrics, or a permutation test on the rankings, would settle the main worry. I would not call the paper misleading; it is a careful empirical study with a reasonable but provisional conclusion.\n\nWho it is for: anyone working on layout adjustment or graph drawing evaluation. It deserves a serious referee, but not publication without the robustness checks. I would send it to review with a request for sensitivity analysis and artifacts.","headline":"Useful first large benchmark and metric taxonomy for overlap removal, but the VPSC/Wordle-L top-two claim rests on one hand-picked metric per criterion with no sensitivity analysis.","tokens_in":9073,"tokens_out":1757,"would_cite":true,"duration_ms":19226,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VPSC and R Wordle-L produce the best overlap-free graph layouts in a five-criterion comparison of eight algorithms.","keywords":["node overlap removal","graph drawing","layout adjustment","quality metrics","mental map preservation","algorithm comparison","graph visualization"],"falsifier":"Rerun the same eight algorithms on the same 854 graphs using the non-selected within-class metrics (for example gs_ch_sd for global shape, nm_knn for node movement, and sp_bb_a for spread) and check whether VPSC and R Wordle-L still lead; if another algorithm tops the revised ranking, the representative-metric choice, not the algorithms themselves, drives the conclusion.","tokens_in":1457,"feed_emoji":"📊","tokens_out":1595,"duration_ms":57550,"temperature":0.7,"pith_summary":"This paper sets out to settle a practical question for visualization designers: given an initial graph layout in which nodes overlap, which post-processing algorithm removes the overlap while best preserving the layout a user has in mind? It reviews 21 quality metrics from the literature, groups them into five quality criteria, and selects one representative metric per criterion, then runs eight state-of-the-art overlap-removal algorithms on 854 graphs under that common measurement protocol. The conclusion is that VPSC and R Wordle-L deliver the best overall quality, with R Wordle-L superior on preserving global shape and VPSC far faster. If the comparison is right, designers have a clear default choice depending on whether speed or shape fidelity matters more. The paper also gives the community a reusable evaluation protocol: future algorithms can be scored on the same five metrics and compared directly.","feed_headline":"VPSC and R Wordle-L produce the best overlap-free layouts","feed_subtitle":"Eight algorithms, five quality metrics, and 854 graphs settle a practical choice for visualization designers.","key_machinery":"The argument is carried by a five-metric evaluation instrument, one representative per quality criterion: oo_nni (normalized number of inversions, capturing orthogonal-order preservation), sp_ch_a (ratio of convex-hull areas, capturing spread minimization), gs_bb_iar (improved bounding-box aspect-ratio deviation, capturing global-shape preservation), nm_dm_imse (mean squared node displacement after a shift-and-scale alignment, capturing node-movement minimization), and el_rsdd (relative standard deviation of Delaunay edge-length ratios, capturing edge-length preservation). This metric set, combined with a fixed experimental protocol using FM3 and SFDP initial layouts and eight algorithms on 854 graphs, is what makes the ranking of algorithms comparable.","core_discovery":"The central claim is that, among eight overlap-removal algorithms evaluated on a uniform set of five quality metrics over 840 synthetic and 14 real-world graphs, VPSC and R Wordle-L dominate on quality: VPSC matches or nearly matches the best on all criteria except global shape, where R Wordle-L is better, while VPSC requires substantially less running time. Uniform scaling scores best on four of the five metrics but is deemed unsatisfactory because it inflates the drawing area. PFS' and PRISM are intermediate, and GTREE suffers from large node movements on synthetic graphs. Across random, tree, small-world, and scale-free graphs the relative results do not change significantly, so the recommendation is not graph-type specific.","pith_inferences":["The paper leaves the relative weight of the five criteria to the designer; a task-weighted composite score could change the VPSC-versus-R Wordle-L trade-off for a specific application.","Because the selected metrics are proxies, the natural robustness check is to rerun the comparison with the discarded within-class metrics such as gs_ch_sd or nm_knn; the paper's reported correlations suggest they behave similarly but not identically.","The conclusion about graph-type independence was tested only on graphs up to 1,000 nodes with one initial layout family per type, so extrapolation to larger graphs or other layout algorithms is an extension of the paper's claim.","A designer who cares about local neighborhood preservation rather than raw displacement might rank GTREE differently, since GTREE did well on real-world graphs but poorly on the movement-distance metric."],"forward_implications":["Designers who need fast overlap removal on larger graphs should prefer VPSC, since it matches R Wordle-L on four of the five quality criteria and runs much faster.","When preserving the global silhouette of the original layout matters most, R Wordle-L is the better choice.","Uniform scaling should not be used in practice despite its top metric scores, because it expands the drawing area too much.","Future overlap-removal papers can report the five representative metrics so their results are directly comparable with this study.","The ranking appears stable across random, tree, small-world, and scale-free graphs, so algorithm choice need not depend on graph type."],"supporting_citations":[{"why":"Supplies the VPSC algorithm that the comparison identifies as the best overall quality-and-speed choice.","marker":"[3]"},{"why":"Supplies the R Wordle-L algorithm and several metrics (oo_ni, sp_ch_a, gs_ch_sd, nm_dm_me) used or considered in the evaluation.","marker":"[17]"},{"why":"Supplies the PRISM algorithm, the el_rsdd and nm_d metrics, and the SFDP baseline layout used for the real-world graphs.","marker":"[5]"},{"why":"Defines the layout-adjustment problem, supplies the PFS algorithm, and contributes foundational metrics such as oo_o and sp_bb_a.","marker":"[15]"},{"why":"Supplies the FTA algorithm and metrics oo_kt, sp_bb_na, and nm_mn.","marker":"[11]"},{"why":"Supplies the GTREE algorithm and the nm_knn metric.","marker":"[16]"},{"why":"Supplies the spring-algorithm approach and metrics gs_bb_ar, el_r, and sp_bb_l1ml.","marker":"[12]"},{"why":"Provides the OGDF graph generators used to create the 840 synthetic graphs.","marker":"[2]"},{"why":"Provides the Graphviz real-world test suite used as a second dataset.","marker":"[6]"}],"fun_headline_variants":["VPSC and R Wordle-L dominate overlap-removal quality","Study: VPSC and R Wordle-L win overlap removal","Overlap removal: VPSC and R Wordle-L outperform","854 graphs rank VPSC and R Wordle-L best for overlap","VPSC fastest, R Wordle-L best shape in overlap removal"],"cache_read_input_tokens":11392,"weakest_assumption_plain":"The entire ranking assumes the five selected metrics are faithful, comparable proxies for the five quality criteria; choosing a different reasonable metric within a class could change which algorithm is declared best.","fun_headline_variants_meta":{"raw":{"variants":["VPSC and R Wordle-L dominate overlap-removal quality","Study: VPSC and R Wordle-L win overlap removal","Overlap removal: VPSC and R Wordle-L outperform","854 graphs rank VPSC and R Wordle-L best for overlap","VPSC fastest, R Wordle-L best shape in overlap removal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001018,"raw_usage":{"total_tokens":4213,"prompt_tokens":778,"completion_tokens":3435,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":394,"completion_tokens_details":{"reasoning_tokens":3346}},"tokens_in":394,"tokens_out":3435,"duration_ms":27109,"temperature":1.0,"reasoning_tokens":3346,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:18:44.498611+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the same eight algorithms on the same 854 graphs using the non-selected within-class metrics (for example gs_ch_sd for global shape, nm_knn for node movement, and sp_bb_a for spread) and check whether VPSC and R Wordle-L still lead; if another algorithm tops the revised ranking, the representative-metric choice, not the algorithms themselves, drives the conclusion.","supporting_citations":[{"cited_title":"In: International Symposium on Graph Drawing","cited_arxiv_id":null,"evidence_quote":"Supplies the VPSC algorithm that the comparison identifies as the best overall quality-and-speed choice."},{"cited_title":"Computer Graph- ics Forum31(3), 1135–1144 (2012)","cited_arxiv_id":null,"evidence_quote":"Supplies the R Wordle-L algorithm and several metrics (oo_ni, sp_ch_a, gs_ch_sd, nm_dm_me) used or considered in the evaluation."},{"cited_title":"Journal of Graph Algorithms and Applications14(1), 53–74 (2010)","cited_arxiv_id":null,"evidence_quote":"Supplies the PRISM algorithm, the el_rsdd and nm_d metrics, and the SFDP baseline layout used for the real-world graphs."},{"cited_title":"Journal of Visual Languages & Computing6(2), 183–210 (1995)","cited_arxiv_id":null,"evidence_quote":"Defines the layout-adjustment problem, supplies the PFS algorithm, and contributes foundational metrics such as oo_o and sp_bb_a."},{"cited_title":"Information Sciences177(14), 2821 – 2844 (2007)","cited_arxiv_id":null,"evidence_quote":"Supplies the FTA algorithm and metrics oo_kt, sp_bb_na, and nm_mn."},{"cited_title":"In: International Symposium on Graph Drawing and Network Visualization","cited_arxiv_id":null,"evidence_quote":"Supplies the GTREE algorithm and the nm_knn metric."},{"cited_title":"In: Proceedings of the 2005 Asia-Pacific Symposium on Information Visualisation - Volume 45","cited_arxiv_id":null,"evidence_quote":"Supplies the spring-algorithm approach and metrics gs_bb_ar, el_r, and sp_bb_l1ml."},{"cited_title":"In: Tamassia, R","cited_arxiv_id":null,"evidence_quote":"Provides the OGDF graph generators used to create the 840 synthetic graphs."},{"cited_title":"Software: practice and experience30(11), 1203–1233 (2000)","cited_arxiv_id":null,"evidence_quote":"Provides the Graphviz real-world test suite used as a second dataset."}],"review_version":1}