{"id":"b9ae68d2-7b0b-41f5-a6ca-7d6d8087bc1c","arxiv_id":"2504.20369","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A perception-aware sampler, built from saliency weights plus density and coverage, yields scatterplot samples with higher measured perceptual similarity than prior methods, and a compressed variant is up to 100x faster.","lead":"This paper introduces scatterplot sampling methods that pick data points so the downsized plot keeps the visual features humans actually notice, using saliency maps as a stand-in for attention. A fast approximate variant, ApproPAwS, produces plots of similar judged quality up to 100 times faster on large datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative evaluation is self-referential: PAwS is built from DVS saliency weights and scored by DVS saliency similarity, so the reported outperformance may be an artifact of the shared model; the only independent check, a 17-participant user study, is too small to carry the central claim.","rationale":"The reader's weakest assumption is exactly the load-bearing concern I identify: the paper's quantitative evaluation is circular with respect to the DVS saliency model, and the paper explicitly declines to test whether that model captures human perception of scatterplots. This is not a matter of internal inconsistency—the method is clearly specified, the experiments are extensive, and the runtime claims are plausible—but the headline claim of 'perceptual similarity' and 'validating our quantitative findings' depends on an external correspondence between DVS saliency and human attention that the paper does not establish. The user study is genuine independent evidence, but it is underpowered for the strength of the claim: 17 participants, no inferential statistics, and Blue-noise slightly preferred in the best-overall category. ApproPAwS's fidelity claim is even less externally supported because it was not in the user study. I agree with the reader's conditional verdict: the paper deserves publication only if the perception proxy is validated or the claims are softened. The concrete eye-tracking test would settle whether the concern actually lands; if PAwS retains its ranking under human fixations, the circularity concern is mitigated and the central claim stands.","tokens_in":35070,"tokens_out":2639,"duration_ms":30323,"concrete_test":"Pre-register an eye-tracking study with at least 60 participants: for each of the six datasets, show the original scatterplot alongside samples from PAwS, ApproPAwS (medium compression), Blue-noise, Max-Min, and Random, under free viewing plus one pattern-report task. Compute human-fixation saliency maps for each stimulus and score each sample against the original using AUC-Judd and NSS. If PAwS does not rank at or near the top on human fixation similarity, statistically equivalent to Blue-noise, then the quantitative claims are artifacts of the shared DVS saliency model. A cheaper complementary check: recompute all five evaluation metrics using an independent saliency model not used for sampling and compare the method rankings.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that PAwS produces scatterplot samples with higher perceptual similarity than prior methods rests on treating DVS saliency maps as a faithful proxy for human perception. That premise is not tested by the quantitative evaluation, because the same model family is used on both sides of the comparison. PAwS weights are computed from aggregate DVS saliency augmented with density (Sec. 3.1, w_p = max(q_s, gamma*q_d)), and all five evaluation metrics (SSIM, CC, SIM, JSD, EMD) compare DVS saliency maps of the sample against DVS saliency maps of the original data (Sec. 5). A method optimized against DVS is therefore being scored by DVS; the reported consistent quantitative outperformance may reflect fitting the metric rather than preserving what humans perceive. The paper explicitly disclaims evaluating saliency-model validity in Sec. 2.4, and the user study is the only external check. That study has 17 participants and roughly 50 trials per comparison pair, reports no statistical tests, and in the 'best overall' category Blue-noise was slightly preferred over PAwS. The ApproPAwS claim of 'minimal loss in visual fidelity' is supported only by the same DVS-based metrics; ApproPAwS was not included in the user study. Thus the load-bearing premise—that DVS saliency approximates human attention in scatterplots—is the least secure part of the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces perception-aware sampling for scatterplots. It proposes a perception-augmented database that stores aggregated Data Visualization Saliency (DVS) maps, augments them with density information to define per-point perception weights w_p = max(q_s, gamma*q_d), and then runs a greedy Max-Min-style algorithm (PAwS) that selects points maximizing weight times distance to the current sample. A second algorithm, ApproPAwS, uses a quad-tree compression of the canvas into boxes with approximately uniform data and similar perception weights, then samples representative points from boxes instead of the true data, claiming up to 100x speedups with minimal perceptual loss. The paper evaluates PAwS and ApproPAwS against Random, DBS, Max-Min, VAS, and Blue-noise on six datasets using five saliency-map similarity metrics (SSIM, CC, SIM, JSD, EMD) and reports a 17-participant user study in which PAwS is often preferred.","tokens_in":35391,"tokens_out":4579,"duration_ms":49379,"significance":"If the perception premise is externally validated, the work is a meaningful step: it is, to my knowledge, a clean and modular proposal for making sampling objectives perception-aware rather than purely geometric or statistical. PAwS is a simple O(kn) greedy extension of Max-Min, ApproPAwS addresses scalability in a principled way, and the paper is honest in Section 2.4 that it treats DVS as a black-box component. The evaluation design, with five metrics, six datasets, and aggregate saliency over multiple visual configurations, is thorough on its own terms. However, the significance of the central claim is conditional on the DVS saliency model being a faithful proxy for human perception of scatterplots, and that premise is not independently established in the paper.","major_comments":[{"comment":"The quantitative evaluation is self-referential: the five metrics (SSIM, CC, SIM, JSD, EMD) all compare DVS saliency maps of the sample against DVS saliency maps of the original data (Section 5, Figure 18b), while PAwS derives its perception weights from the same aggregate DVS saliency with w_p = max(q_s, gamma*q_d) (Section 3.1). Optimizing and scoring with the same model family can produce systematic advantages that reflect fitting DVS rather than preserving what humans perceive. Section 2.4 explicitly disclaims evaluating how well saliency models capture visual attention, so the user study is the only independent check in the paper. The central claim of consistent outperformance will be load-bearing only if the paper adds external validation, such as eye-tracking or a larger user study covering all methods and ApproPAwS, or a sensitivity analysis that varies the saliency model to show the conclusions are not an artifact of the DVS choice.","section":"Section 5 and Section 3.1"},{"comment":"The user study is too weak to carry the central perception claim alone. It has 17 participants and roughly 50 trials per comparison pair, reports no significance tests or confidence intervals, and in the 'best overall' category Blue-noise is slightly preferred over PAwS, which is the opposite of the paper's overall narrative. Since this is the only external evidence connecting the sampling objective to human perception, the paper should report per-dataset effect sizes with statistical tests (including multiple-comparison correction), and the inconclusive best-overall result should be discussed rather than minimized.","section":"Section 6.1, Figure 15"},{"comment":"The claim that ApproPAwS achieves 'minimal loss in visual fidelity' is supported only by the same DVS-based metrics used to construct the method, and ApproPAwS was not included in the user study. In addition, the compression thresholds for ACSI are adjusted differently from the other datasets 'to observe notable differences across compression schemes' (Section 6.2 and Appendix A), which is a post hoc selection that weakens the robustness claim. The paper should provide a sensitivity analysis over lambda, sigma, and C, and ideally independent evaluation of ApproPAwS outputs, before claiming comparable fidelity to PAwS.","section":"Section 6.2, Figure 16, Appendix A"},{"comment":"For Hidden Correlation, the largest dataset (~3.5M points), the density component of the perception weight was not computed with the kernel density estimator but approximated by rendering the data with a small point size and opacity. Because Hidden Correlation is also a dataset where PAwS shows particularly large reported gains, this substitution should be disclosed prominently in the main evaluation section and its effect on the density term in w_p should be analyzed.","section":"Section 6, footnote 8"},{"comment":"Algorithm 2 as written is not well-defined: line 6 initializes D' with 'a random point in P', but P is the set of partitions/boxes, not data points, so the distance d(r,x) in line 8 and the membership test in line 10 are not defined as printed. This should be corrected (e.g., initialize from the representative set R) before the ApproPAwS runtime and results can be reproduced.","section":"Algorithm 2"}],"minor_comments":[{"comment":"The Jensen-Shannon Divergence formula has a typo: the second term should be D_KL(q_D || mu), not D_KL(p_D' || mu), otherwise the expression is not symmetric as claimed.","section":"Section 5, JSD formula"},{"comment":"The caption states that 'Vas is slightly preferred in the best overall category', while the body text says Blue-noise slightly overtakes PAwS in that category; the reported result should be made consistent between the caption and the text.","section":"Figure 15 caption"},{"comment":"The adaptive setting of gamma via a sigmoid mapping of density variance is described only verbally; the paper should give the concrete mapping and any default parameter values so that the method is reproducible.","section":"Section 3.1"},{"comment":"The labeling of low, medium, and high compression in Figure 10 is confusing because the lower-right panel is described as the highest compression in the caption but the relative order of the panels is not labeled directly in the figure; please add explicit labels.","section":"Figure 10"},{"comment":"Line 10 says 'where v in b', but v is a representative point, not a box; please clarify that b is the box from which v was originally drawn.","section":"Algorithm 2"}],"recommendation":"major_revision","confidential_remarks":"The core algorithmic contribution is sound and the paper is transparent about its scope, but the evaluation's quantitative core is self-referential with respect to the DVS saliency model, and the independent user study is underpowered. I would not reject the paper on this basis; the direction is novel and the ApproPAwS speedups are interesting. However, the revision needs to either add external validation of the saliency premise or substantially reframe the claims so that they are about optimizing DVS saliency rather than human perception. The novelty claim of being the first to model perception in scatterplot sampling is somewhat strong given the prior work by Alabi and Wu [6], though the authors do cite that work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a genuine new idea: using saliency maps as a sampling objective for scatterplots, with two concrete algorithms (PAwS and its approximate variant ApproPAwS) and a reasonable evaluation. Second, the evaluation has a real circularity problem: PAwS is built from DVS saliency weights and scored by DVS saliency similarity, so the quantitative outperformance may just be fitting the metric. The user study is the only independent check, and it's too thin to carry the load.\n\nWhat's good: the problem is well-motivated, the algorithms are simple and clearly described (O(kn) greedy max-min with perception weights; quad-tree compression for ApproPAwS), and the experimental setup covers six datasets with different shapes and sizes. The paper is honest about scope—Section 2.4 explicitly says they are not evaluating how well saliency models capture attention. The qualitative examples (Figure 1) are convincing. The user study, while small, does show that humans often prefer PAwS to Random, DBS, VAS, and Max-Min, which is real evidence that something is working.\n\nWhere it's soft: the circularity is not minor. All five quantitative metrics compare DVS saliency maps, and DVS is the same model used to compute the perception weights. So the \"consistently outperform\" claim is conditional on DVS being a faithful proxy. The paper declines to test that, which is a legitimate scope choice, but then it can't claim to have validated human perception. The user study is the only external anchor: 17 participants, no statistical tests, and in the \"best overall\" category Blue-noise slightly beat PAwS. ApproPAwS's \"minimal loss in visual fidelity\" is supported only by the same DVS metrics, since it wasn't in the user study. There are also minor issues: ACSI compression thresholds were adjusted after inspection, the density weights for the largest dataset were approximated using a small-point rendering, and no code is released.\n\nThe central argument—that optimizing for saliency plus density, with coverage, produces samples that look better—plausibly holds, but the paper overstates the evidence. The flaws are addressable: add an independent human-judgment evaluation with more participants, report statistical tests, fix thresholds before evaluation, and release code.\n\nI'd send this to serious reviewers. It's a useful contribution to the visualization-sampling subfield, and with a stronger external check it could be a solid paper. Recommendation: major revision.","headline":"Useful new sampling method for scatterplots, but the headline outperformance claim rests on a self-referential DVS evaluation and a too-thin user study.","tokens_in":35931,"tokens_out":2555,"would_cite":true,"duration_ms":24955,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Perception-aware sampling for scatterplots selects points by weighting saliency and density inside a greedy farthest-point heuristic, yielding samples that the paper shows are perceptually closer to the original data than five prior…","keywords":["perception-aware sampling","scatterplots","saliency maps","data sampling","visualization","approximate visualization","farthest-point sampling","perceptual similarity"],"falsifier":"Run an eye-tracking study where participants view the full scatterplot and then pairs of samples; if fixation maps on PAwS samples are not systematically closer to fixation maps on the full data than the baselines' fixation maps are, the perception-preservation claim fails. A cheaper test is to swap the DVS model for a different saliency model or for human-fixation maps inside the pipeline and re-run the same five metrics; if the reported gains vanish, PAwS's advantage is an artifact of self-evaluation with the same model family.","tokens_in":34857,"feed_emoji":"📊","tokens_out":10166,"duration_ms":90425,"temperature":0.7,"pith_summary":"Scatterplot down-sampling routinely hides the very structure—small clusters, trends, outliers—that motivated drawing the plot in the first place, and existing sampling methods optimize data properties such as density or coverage rather than how people see. The paper argues that sampling should be perception-aware: weight each candidate point by a saliency map, which predicts where viewers look, augmented by local density, and then greedily select points that maximize the product of this weight and distance to the already-selected set. It claims that the resulting PAwS samples are perceptually closer to the original full-data visualization than Random, Density-Biased Sampling, Max-Min, Visualization-Aware Sampling, and Blue-noise samples across six datasets and five similarity metrics, and that the approximate variant ApproPAwS is up to 100x faster with minimal perceptual loss. If the claim holds, analysts could work with smaller and faster samples without giving up the patterns they are trying to read.","feed_headline":"Saliency-guided sampling beats five methods at preserving scatterplots","feed_subtitle":"The new sampler matches what humans see in the full data with far fewer points, and runs up to 100x faster.","key_machinery":"The load-bearing object is the per-point perception weight $w_p = \\max(q_s, \\gamma \\cdot q_d)$, which turns a saliency heatmap into a sampling score: $q_s$ says where attention is predicted to go, $q_d$ adds the density information that saliency models miss, and the $\\max$ ensures either can elevate a point. PAwS plugs this weight into a greedy farthest-first traversal, scoring candidates as $w_y \\cdot \\min_{x \\in D'} d(y, x)$; the multiplicative combination is what balances perception with coverage, so the sample does not collapse onto attention hotspots. ApproPAwS adds a second mechanism: a quad-tree partition splits any cell whose Chamfer distance to uniform random points exceeds $\\lambda$ or whose perception-weight variance exceeds $\\sigma$, yielding a compressed box representation from which representative points are drawn uniformly, so the approximate visualization never touches the original data.","core_discovery":"The central claim is that the best scatterplot sample is the one that distorts the original image's saliency map least, and that this objective can be chased with a greedy farthest-point heuristic. PAwS assigns each point a perception weight $w_p = \\max(q_s, \\gamma \\cdot q_d)$, where $q_s$ is the saliency value from the Data Visualization Saliency model aggregated across 16 rendering configurations and $q_d$ is a kernel-density estimate, with $\\gamma$ set adaptively from the variance of densities. It then iteratively adds the point maximizing $w_y \\cdot \\min_{x \\in D'} d(y, x)$, a multiplicative score that forces the sample to favor salient and dense regions while still covering the whole point cloud. The paper reports that PAwS consistently outscores five baselines on SSIM, CC, SIM, 1-JSD, and EMD across six datasets, reaches a given perceptual similarity at far smaller sample sizes, and that ApproPAwS, sampling from a quad-tree-compressed representation, matches PAwS's saliency scores while running up to 100x faster on a 3.5-million-point dataset.","pith_inferences":["The evaluation scores samples with the same saliency-model family that builds the sampling weights; a human-fixation benchmark would give a fairer test, and the paper's own 17-participant study shows blue-noise slightly ahead in the 'best overall' category.","PAwS's advantage is concentrated on datasets with hidden or mixed structure, such as the synthetic Hidden Correlation; on uniformly clustered data like MNIST all methods converge, suggesting perception-aware sampling matters most when structure is easily missed.","ApproPAwS's boxes could serve as a spatial index for interactive exploration, allowing re-sampling at new zoom levels or filtered regions from the same compressed representation without recomputing saliency.","The aggregation of saliency across rendering configurations (currently the per-pixel maximum) is an arbitrary choice; testing other fusion rules or replacing DVS with another model would reveal how much of the gain depends on that choice."],"forward_implications":["PAwS needs fewer points to reach a target perceptual similarity: with 844 points it averages a 0.75 SSIM score where VAS needs roughly five times as many points.","PAwS is orders of magnitude faster than VAS and blue-noise on large data, reducing sample generation from hours to minutes on a 3.5-million-point dataset.","ApproPAwS's compressed representation lets analysts generate approximate visualizations without re-reading the original data, with reported speedups up to 100x and saliency-similarity scores close to PAwS.","Because the saliency model is a plug-in black box, future improvements in visualization saliency prediction should carry over directly to better perception-aware samples."],"supporting_citations":[{"why":"Supplies the DVS saliency model that generates the saliency maps from which perception weights are derived.","marker":"[67, 96]"},{"why":"Defines the Visualization-Aware Sampling baseline whose coverage-only objective PAwS is designed to beat.","marker":"[75]"},{"why":"Provides the greedy farthest-first (GMM) algorithm that PAwS adapts by inserting perception weights.","marker":"[88]"},{"why":"Defines SSIM, the primary image-based metric used to compare sample and original saliency maps.","marker":"[102]"},{"why":"Supplies the distribution-based saliency metrics (CC, SIM, JSD, EMD) and their recommended evaluation methodology.","marker":"[18]"},{"why":"Source of the DBS and blue-noise baseline implementations, the sample-size schedule, and the prior user-study comparison framing.","marker":"[107]"},{"why":"Describes the blue-noise sampling algorithm included as a state-of-the-art baseline.","marker":"[106]"},{"why":"Establishes that humans often fail to perceive small data perturbations, the premise on which ApproPAwS's approximate visualizations rest.","marker":"[44]"}],"fun_headline_variants":["Saliency-guided scatterplot sampling matches human perception","Perception-aware scatterplot sampling wins on five metrics","Scatterplot sampling that preserves saliency at 100x speed","Human-perception-aware scatterplot sampling: 100x faster","Saliency-based scatterplot sampling beats five baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's quantitative case assumes that the DVS saliency model's predicted attention maps faithfully mirror what humans actually see in scatterplots, because all five evaluation metrics compare saliency maps produced by that same model family and the paper explicitly declines to test this assumption.","fun_headline_variants_meta":{"raw":{"variants":["Saliency-guided scatterplot sampling matches human perception","Perception-aware scatterplot sampling wins on five metrics","Scatterplot sampling that preserves saliency at 100x speed","Human-perception-aware scatterplot sampling: 100x faster","Saliency-based scatterplot sampling beats five baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000764,"raw_usage":{"total_tokens":3438,"prompt_tokens":1042,"completion_tokens":2396,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":2316}},"tokens_in":658,"tokens_out":2396,"duration_ms":19811,"temperature":1.0,"reasoning_tokens":2316,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:31:08.275501+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run an eye-tracking study where participants view the full scatterplot and then pairs of samples; if fixation maps on PAwS samples are not systematically closer to fixation maps on the full data than the baselines' fixation maps are, the perception-preservation claim fails. A cheaper test is to swap the DVS model for a different saliency model or for human-fixation maps inside the pipeline and re-run the same five metrics; if the reported gains vanish, PAwS's advantage is an artifact of self-evaluation with the same model family.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Visualization-Aware Sampling baseline whose coverage-only objective PAwS is designed to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the greedy farthest-first (GMM) algorithm that PAwS adapts by inserting perception weights."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the DBS and blue-noise baseline implementations, the sample-size schedule, and the prior user-study comparison framing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that humans often fail to perceive small data perturbations, the premise on which ApproPAwS's approximate visualizations rest."}],"review_version":1}