{"id":"5aae96b1-a669-4216-afa5-6734d2ac09fb","arxiv_id":"2412.07605","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"One-shot pruned graph and weight masks, when gradually denoised with gradient- and degree-based swapping, yield graph lottery tickets that are sparser and faster to find than iterative magnitude pruning.","lead":"This paper proposes FastGLT, a method that finds sparse graph neural networks and sparse input graphs in a single pruning pass followed by a short denoising phase. It reports winning tickets that are sparser and found 1.7 to 44 times faster than iterative magnitude pruning methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'higher sparsity' headline rests on an unspecified extreme-sparsity success criterion; without a tolerance, the reported 1.32%–45.62% WS and 7.49%–22.71% GS gains are not reproducible or falsifiable.","rationale":"After reading the full manuscript, I find the most load-bearing weakness to be the under-specified 'extreme sparsity' success criterion, not the Sec. 2.3 Cora-based denoising pattern. The method's empirical results already span six datasets and four backbones, which provides some support for generalizability; moreover, even if the exact gradient/degree pattern is not universal, the denoising procedure could still improve tickets through other mechanisms, so the central claim does not strictly depend on that pattern. In contrast, the quantitative claim of 'sparser' tickets is directly computed from extreme-sparsity numbers whose success threshold is never defined. Sec. 2.2 gives only a qualitative definition; Appendix C.5 describes an arithmetic search with no tolerance, step size, or assurance that baselines were searched identically. The abstract and introduction report improvement ranges that themselves differ (1.32%–45.62% vs. 5.82%–25.48% for WS), which I interpret as a symptom of the same ambiguity. This concern is concrete, fixable, and testable: re-running the search with explicit tolerances and a shared grid would confirm or refute the headline sparsity margins. Since the reader already called for conditional acceptance pending corrections, my read does not change the verdict; the concern is addressable and does not invalidate the method's overall contribution.","tokens_in":17569,"tokens_out":7259,"duration_ms":63566,"concrete_test":"Recompute the extreme-sparsity search on Cora+GCN, Citeseer+GCN, and Ogbn-Arxiv+ResGCN for FastGLT, UGS, WD-GLT, and one-shot pruning using a pre-registered success criterion: mean test accuracy ≥ baseline mean minus one standard deviation (and also ≥ baseline mean − 0.5% as a sensitivity check), with a common grid step of 1% and independent seeds. If the claimed sparsity improvement margins (1.32%–45.62% WS; 7.49%–22.71% GS) move by more than a couple of percentage points under either tolerance, the reported sparser-ticket advantage is an artifact of the unspecified criterion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 2.2 defines extreme graph/weight sparsity as the maximum sparsity at which a method 'successfully identifies GLTs,' with a GLT requiring test accuracy at least that of the dense baseline. However, the operational criterion is never quantified: no accuracy tolerance (e.g., within one standard deviation, within 0.5%, or strictly ≥ baseline mean) is given, and Appendix C.5 only states that sparsity levels are searched 'in an arithmetic sequence until a lottery ticket can no longer be found.' The step size of that sequence and whether the same protocol is applied to UGS, WD-GLT, and one-shot pruning are not reported. This matters because the paper's central quantitative claim—sparser tickets than IMP-based methods (1.32%–45.62% WS improvement, 7.49%–22.71% GS improvement, Tables 2, 4, 8)—is computed from these extreme-sparsity values. If FastGLT's accuracy declines gracefully while UGS/WD-GLT drop sharply, the reported extreme sparsity and hence the 'higher sparsity' conclusion depend heavily on the unstated tolerance and grid. The concern is not that the method fails, but that the headline sparsity advantage is not falsifiable from the information given; a different reasonable tolerance could shrink or erase the reported gaps. This is the most load-bearing weakness because it directly governs the 'sparser' half of the paper's central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FastGLT, a one-shot pruning and denoising framework for finding graph lottery tickets. Instead of iterative magnitude pruning, FastGLT first prunes graph and weight masks to an intermediate sparsity in one shot, then gradually swaps low-magnitude retained elements for high-gradient weights and low-degree edges, progressively increasing sparsity to the target. Experiments on six datasets and four GNN backbones compare FastGLT with UGS, WD-GLT, and random pruning, and the authors claim higher weight and graph sparsity, 1.7-44x speedups over IMP-based methods, and 95%+ MAC savings. The central conclusion is that one-shot tickets can be denoised into tickets that are sparser than, and perform comparably to, IMP-based tickets.","tokens_in":17942,"tokens_out":7967,"duration_ms":72012,"significance":"The fast-track hypothesis is interesting and potentially useful: if one-shot tickets are structurally close to IMP tickets, replacing k rounds of IMP with one-shot pruning plus denoising could make graph lottery ticket search substantially cheaper. I credit the authors for releasing code, reporting standard deviations over five trials, and including ablation studies on the initial ticket type, denoising interval, and scheduler decay. If the quantitative claims withstand scrutiny, the practical contribution is solid. However, the two quantitative pillars of the paper—higher sparsity and faster search—are not yet fully substantiated as reported: the extreme-sparsity criterion is not operationally defined, and the wall-clock normalization in Table 1 is internally inconsistent. These issues are fixable and do not by themselves invalidate the method, but they must be resolved before the headline numbers can be trusted.","major_comments":[{"comment":"The extreme-sparsity measurements that support the headline 'higher sparsity' claim are not operationally defined. Definition 1 requires test accuracy at least that of the dense baseline, but because accuracy is stochastic, a tolerance (e.g., within one standard deviation, within 0.5%, or strictly ≥ the baseline mean) is needed. Appendix C.5 only says that sparsity levels are searched 'in an arithmetic sequence until a lottery ticket can no longer be found,' without specifying the step size or whether UGS and WD-GLT are evaluated under the same search protocol. Since Tables 2, 4, and 8 report extreme sparsities on the basis of this undefined criterion, the reported 1.32%-45.62% weight-sparsity and 7.49%-22.71% graph-sparsity improvements are not falsifiable from the information given; a different tolerance or grid step could materially change them.","section":"§2.2, Appendix C.5, Tables 2/4/8"},{"comment":"The 'Relative Time' column is computed inconsistently, so the 1.7-44x speedup claim is not reproducible as reported. For GCN FastGLT, 1.63x is the mean of the three per-dataset ratios (34.9/21.4, 89.7/41.4, 1366.2/1217.3), while for GIN UGS the reported 28.6x is the sum of the three ratios (39.8/7.3, 61.0/8.6, 141.7/8.8), and for GAT FastGLT 4.8x is again a sum (525.9/333.1, 528.7/284.1, 1270.3/920.7). Obs.4's statement that 'UGS requires 4.0-28.6x' therefore mixes averaged and summed quantities; the table should be recomputed in a single consistent way and the speedup ranges restated accordingly.","section":"Table 1 and §4.4"},{"comment":"The headline improvement ranges are mutually inconsistent. The abstract and Obs.1 report weight-sparsity gains of 1.32%-45.62% and graph-sparsity gains of 7.49%-22.71%, while the contribution bullet in §1 reports 5.82%-25.48% and 3.65%-17.48%. Because these ranges are the paper's main quantitative claim, the authors should reconcile them and specify exactly which table and which baseline each range refers to.","section":"Abstract, §1, §4.2"},{"comment":"The denoising mechanism is motivated by a two-part empirical pattern observed only on Cora: relative to IMP, one-shot pruning removes weights with relatively high gradients and edges with relatively low degree, and Eqs. 5-6 instantiate exactly these signals for every dataset and backbone. The paper should report the same gradient/degree diagnostics for at least one additional small graph and one OGB graph, or explicitly state that transferability is an assumption; otherwise the mechanism's generality is a risk rather than a validated premise.","section":"§2.3 and §3.2"}],"minor_comments":[{"comment":"Line 6 computes the intermediate sparsity as sinm ← Υ(stgt), but Υ is defined in §3.2 as the denoising scheduler; the intermediate sparsity should be computed with Ψ(s)=s−αs^β from §3.1.","section":"Algorithm 1, line 6"},{"comment":"The formula for ωθ uses stgt_g in both ωg and ωθ, but the weight-sparsity increment should presumably use stgt_θ; this appears to be a typo and should be corrected.","section":"Eq. 6 and surrounding text"},{"comment":"In the GCN block of Table 7, FastGLT reports accuracy 79.11 for both Citeseer and PubMed; the Citeseer value duplicates the PubMed value and is implausible relative to the 70.51 Citeseer baseline. This data error should be corrected.","section":"Table 7"},{"comment":"The denoising scheduler Υ(µ)=τ(1−µ/µend)^κ contains an initial ratio τ that is never specified in Table 3 or in the sensitivity analysis in Appendix C.3; please report the value of τ used in all experiments.","section":"§3.2 and Table 3"},{"comment":"The text uses 'wall-lock time' in the Figure 1 caption and 'wall-lock' in Figure 5; this should read 'wall-clock.' In addition, Table 1's header 'Relative Time (s)' is a unitless ratio and should be labeled accordingly.","section":"Table 1 and Figures 1/5"},{"comment":"The phrase 'task-itrelevant loss function' should read 'task-irrelevant loss function.'","section":"§3.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a look. It makes a real empirical case that one-shot joint graph/weight pruning plus a short denoising phase can find winning tickets nearly as well as IMP, at a fraction of the wall-clock cost. The specific mix—one-shot pruning to an intermediate sparsity, then iterative gradient/degree-based swapping of a small set of pruned elements—is new relative to the GLT literature, and the experiments are broad: 6 datasets, 4 backbones, ablations, and a comparison to UGS and WD-GLT. The ablations showing that starting from one-shot tickets matters (vs. random tickets) support the 'fast track' hypothesis.\n\nThe soft spots are real but mostly fixable. The biggest one is the operational definition of 'extreme sparsity.' The paper says it is the maximum sparsity at which a GLT is found, where a GLT requires test accuracy at least the dense baseline, but there is no tolerance and no grid step. The appendix just says an arithmetic sequence is searched. Since the headline claim—higher weight and graph sparsity than IMP-based methods—is computed from these extreme sparsity values, a different reasonable tolerance could shrink or erase the reported gap. This needs to be pinned down before the numbers can be trusted.\n\nThere are also internal inconsistencies: the abstract and contributions report different improvement ranges; Table 1 lists UGS on GIN as 28.6× relative time, which does not match the raw times in the same table (average about 9.5×); Algorithm 1 uses the denoising scheduler Υ to compute intermediate sparsity where the text uses Ψ; and Eq. (6) defines ω_θ with graph sparsity instead of weight sparsity. These are minor typos except the Table 1 one, which looks like a reporting error.\n\nOne more thing: the gradient/degree motivation is only visualized for Cora. The OGB results suggest the pattern generalizes, but a second dataset in that figure would be more convincing. Some hyperparameters (τ, α, β) are never given explicit defaults.\n\nWho is this for? Researchers working on efficient GNN training or pruning. The method is clearly described and the code is promised. I would send it to peer review, but the authors need to specify the extreme sparsity protocol exactly, correct the Table 1 numbers, and fix the notation errors. I'd cite the method once these are cleaned up.","headline":"A genuinely new fast-track method for graph lottery tickets with broad experiments, but the headline sparsity advantage rests on an unspecified success criterion and the reporting has several inconsistencies.","tokens_in":18421,"tokens_out":4863,"would_cite":false,"duration_ms":42105,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One-shot pruning plus gradient-degree denoising can match iterative IMP at higher sparsity and up to 44x lower cost.","keywords":["graph lottery ticket","one-shot pruning","iterative magnitude pruning","graph sparsification","GNN pruning","gradient-based denoising","edge degree","FastGLT"],"falsifier":"Take a graph from a different family, such as a heterophilic graph or a protein interaction graph, run FastGLT and an iterative magnitude pruning baseline to the same target sparsity, and compare the identities of restored elements. If the pruned weights that IMP keeps do not have consistently higher accumulated gradients, or the pruned edges IMP keeps do not have consistently lower edge degree, then the denoising signal is not the mechanism and the fast-track claim fails on that graph family.","tokens_in":17397,"feed_emoji":"🎫","tokens_out":5699,"duration_ms":56473,"temperature":0.7,"pith_summary":"The paper argues that expensive iterative magnitude pruning is not necessary to find graph lottery tickets. It shows that a single one-shot pruning run produces masks already close to IMP masks, with the residual difference following a consistent pattern: one-shot pruning wrongly removes weights with relatively large gradients and edges with relatively low degree. Based on this, FastGLT prunes once to an intermediate sparsity, then repeatedly swaps low-magnitude kept elements for high-gradient weights and low-degree edges from the pruned set. The reported result is a double win: winning subgraphs and subnetworks with higher weight and graph sparsity than IMP-based methods, found in E+D epochs instead of kE, with 1.7-44x speedups and over 95% MAC savings. If right, this reframes one-shot pruning from a cheap fallback into the main route for graph lottery ticket search.","feed_headline":"One-shot pruning can replace IMP to find sparser GNN tickets","feed_subtitle":"A denoising swap of high-gradient weights and low-degree edges turns cheap one-shot masks into winning tickets.","key_machinery":"The load-bearing mechanism is a two-stage mask pipeline. Stage one trains trainable masks $m_g$ and $m_\\theta$ end-to-end, then thresholds the lowest-magnitude entries at an intermediate sparsity computed by $\\Psi(s)=s-\\alpha s^\\beta$ to produce one-shot masks $M^\\circledcirc_g$ and $M^\\circledcirc_\\theta$. Stage two, the gradual denoising mechanism, works in $\\lceil D/\\Delta T\\rceil$ intervals: at each interval it labels currently kept elements with the smallest magnitudes as noisy, labels pruned weights with the largest accumulated gradients and pruned edges with the smallest edge degrees as potential, and swaps them via $M^{(\\mu+1)}=(M^{(\\mu)}\\setminus M^{(ns)})\\cup M^{(pt)}$, increasing sparsity toward the target by a fixed amount per interval. This swap is what turns a cheap one-shot ticket into a winning ticket.","core_discovery":"The central discovery is that the gap between one-shot and IMP masks is small and structured rather than random. Across sparsity levels, the Hamming distance between one-shot masks and IMP masks stays far smaller than the distance between random masks and IMP masks, and the mistakes one-shot pruning makes are biased in a specific way: compared with IMP, one-shot pruning cuts weights that have higher gradients and edges that have lower degree. The paper treats these as identifiable structural noise that can be corrected. FastGLT therefore uses one-shot masks as a fast track and denoises them: within each denoising interval it identifies currently kept elements with the smallest magnitudes as noisy, discovers pruned weights with the largest accumulated gradients and pruned edges with the smallest edge degrees as potential, and swaps them, gradually raising sparsity to the target. The paper validates this by finding winning tickets at higher extreme sparsity than UGS and WD-GLT across 4 backbones and 6 datasets, with weight sparsity gains of 1.32%-45.62%, graph sparsity gains of 7.49%-22.71%, and 1.7-44x search speedups.","pith_inferences":["The same recipe, prune once and then restore elements selected by a measured mismatch signal, could transfer to other structured pruning settings wherever one-shot and iterative masks differ in a measurable way.","A natural next test is whether the gradient and degree signals identify IMP's choices on graphs with very different degree distributions, such as heterophilic graphs or protein interaction graphs; if not, the potential-component metric would need to become dataset-adaptive.","An implicit consequence is that the method's advantage grows with the number of IMP rounds it replaces, so FastGLT is most attractive at very high target sparsity where iterative magnitude pruning becomes prohibitively expensive.","If the pattern is universal, one-shot prune-and-denoise could replace iterative magnitude pruning in other lottery ticket settings, making lottery ticket search feasible on graphs where IMP's cost is prohibitive."],"forward_implications":["If the fast-track pattern generalizes, graph lottery ticket search no longer needs k rounds of prune-and-retrain; one-shot pruning plus denoising is enough.","FastGLT produces winning tickets with higher extreme sparsity than UGS and WD-GLT: up to 45.62% higher weight sparsity and 22.71% higher graph sparsity on small graphs, and weight sparsity above 70% on large OGB graphs.","The reported efficiency gain is 1.7-44x over IMP-based GLT search with 95.3%-98.6% MAC savings, so sparse GNN training and inference become substantially cheaper.","Because denoising from random tickets loses up to roughly 29% weight sparsity and 24% graph sparsity, the one-shot start is load-bearing: the fast track itself, not just the swapping mechanism, is what makes the method work."],"supporting_citations":[{"why":"Defines the graph lottery ticket problem and supplies UGS, the main IMP-based baseline whose masks and sparsity results FastGLT compares against.","marker":"Chen et al. 2021b"},{"why":"Supplies WD-GLT, the graph-sparsity-aware IMP-based baseline that FastGLT must beat in both sparsity and speed.","marker":"Hui et al. 2023"},{"why":"Establishes the lottery ticket hypothesis that motivates the existence of sparse trainable subnetworks.","marker":"Frankle and Carbin 2018"},{"why":"Provides the gradient-based one-shot pruning perspective that FastGLT builds on and compares with IMP.","marker":"Lee, Ajanthan, and Torr 2018"},{"why":"Supplies the gradient-based sparse training and pruning-and-growing ideas behind the denoising and Inverse Power scheduler.","marker":"Evci et al. 2020"},{"why":"Establishes the magnitude-pruning-with-retraining schedule that IMP-based GLT methods use and that FastGLT avoids.","marker":"Zhu and Gupta 2017"},{"why":"Contributes the Hamming distance mask-comparison tool used to show one-shot masks stay close to IMP masks.","marker":"You et al. 2022"},{"why":"Provides the observation that irreversibly pruned elements contain valuable structure, motivating the gradual denoising swap mechanism.","marker":"Wang et al. 2023d"}],"fun_headline_variants":["One-shot pruning, denoised, beats IMP","Sparse GNN tickets in one pass, not many","FastGLT: denoise one-shot masks for sparser tickets","Cheap masks, high sparsity: winning GNN tickets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method depends on the assumption that one-shot pruning makes the same kind of mistakes on every graph as it does on Cora: it wrongly removes weights that have large gradients and edges whose endpoints have few connections.","fun_headline_variants_meta":{"raw":{"variants":["One-shot pruning, denoised, beats IMP","Sparse GNN tickets in one pass, not many","FastGLT: denoise one-shot masks for sparser tickets","Cheap masks, high sparsity: winning GNN tickets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000559,"raw_usage":{"total_tokens":2720,"prompt_tokens":1068,"completion_tokens":1652,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":684,"completion_tokens_details":{"reasoning_tokens":1583}},"tokens_in":684,"tokens_out":1652,"duration_ms":12641,"temperature":1.0,"reasoning_tokens":1583,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:40:16.183388+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a graph from a different family, such as a heterophilic graph or a protein interaction graph, run FastGLT and an iterative magnitude pruning baseline to the same target sparsity, and compare the identities of restored elements. If the pruned weights that IMP keeps do not have consistently higher accumulated gradients, or the pruned edges IMP keeps do not have consistently lower edge degree, then the denoising signal is not the mechanism and the fast-track claim fails on that graph family.","supporting_citations":[],"review_version":1}