{"id":"8269dc9f-3872-46c7-98cd-2406b1762cb7","arxiv_id":"2412.07188","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GNNs can produce frequency components absent from their input, so their spectral behavior is not dominated by the aggregation filter, and a new benchmark quantifies this across 14 models.","lead":"This paper benchmarks 14 graph neural networks by how well they predict labels built from each frequency band of the graph spectrum, and finds that all models handle the lowest and highest frequencies better than middle ones. The result challenges the common view that a GNN's behavior is set by its neighborhood-aggregation filter, so spectral analyses should treat the whole network, including nonlinearities, as the object of study.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark lacks a no-graph baseline; V-shaped accuracy curves and model rankings may be artifacts of the synthetic label-generation process rather than GNN spectral capabilities.","rationale":"The reader identified the synthetic-label operationalization as the weakest assumption. The most direct testable version is the absence of a no-graph baseline: without it, the benchmark cannot distinguish \"the GNN uses the graph to capture this frequency component\" from \"this frequency component's label pattern happens to be easy or hard for any model.\" This is load-bearing because Section 5.2's central interpretive claim (\"inherent limitation of neighborhood aggregation mechanism\") and the model rankings in Table 1 are the paper's main empirical contributions. I agree with the reader's conditional verdict: the central claim that the filter does not uniquely determine GNN spectral behavior is plausible and supported by the exploratory study, but the benchmark needs a no-graph control and a fix for the vacuous bound in Theorem B.4 (for realistic n, 2(4n/k^2)^{1/n} approaches 2, which exceeds the maximum possible L2 distance between energy distributions, so the stated consistency guarantee is empty). The concrete test above would settle whether the missing baseline invalidates the benchmark's interpretation.","tokens_in":25062,"tokens_out":8350,"duration_ms":91453,"concrete_test":"Run the benchmark protocol of Section 4.1 with a 2-layer MLP baseline (hidden size 64, Adam lr 0.001, 500 epochs, same 60/20/20 transductive split, cross-entropy loss), using only the original node features and no adjacency information, on all six datasets and all frequency bins. Compare the MLP's spectral accuracy curves and Normalized AUAC rankings against Table 1 and Figure 2. If the MLP produces V-shaped curves and its per-bin accuracies are strongly correlated with the GNN curves (e.g., rank correlation above 0.8 across bins), the benchmark does not isolate graph-based spectral processing and the paper's conclusions need revision. If the MLP curves are substantially flatter or uncorrelated, the no-graph concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The benchmark protocol (Section 4.1, Appendix A.2) generates node-classification labels by discretizing entries of graph-Laplacian eigenvectors into bins. Section 5.2 then interprets the resulting V-shaped accuracy curves as evidence that \"GNNs' weakness in capturing the task-relevant information encoded in the middle frequency components reveals an inherent limitation of neighborhood aggregation mechanism.\" This attribution is not justified because the protocol never includes a control model without neighborhood aggregation. The input features are the original node attributes (e.g., BOW text), while the labels are quantized eigenvector entries. Low-frequency eigenvectors are smooth over the graph, high-frequency eigenvectors are alternating, and mid-frequency eigenvectors are neither; these label-structure properties alone can make low and high bins easier or harder for any classifier, including an MLP that completely ignores the adjacency matrix. If a no-graph MLP reproduces the same V-shaped curves, or if the rankings across models are similar to those of an MLP, then the benchmark is measuring the learnability of synthetic labels from node features rather than the frequency-capturing capability of GNNs. The central claim of the paper (that filters do not dominate GNN behavior) is not directly refuted by this gap, but the benchmark's stated purpose and its RQ1/RQ2 conclusions are unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper revisits the widely held view that the neighborhood-aggregation filter dominates GNN behavior in the spectral domain. It first presents an exploratory study in which a GCN is trained to regress target signals concentrated in frequency bands that are absent from the input; the outputs align with the target energy distribution, suggesting that nonlinear layers can create frequency components that the linear filter would suppress. The paper then proposes a benchmark protocol: eigenvectors of the normalized graph Laplacian are binned by eigenvalue, the entries of the binned vectors are discretized into classes, and the accuracy of 14 GNNs in predicting these synthetic labels is plotted against frequency to form 'accuracy curves' and a Normalized AUAC ranking. A theoretical analysis (Theorems 4.2–4.4) is offered to justify that discretization does not significantly perturb the energy distribution. The empirical results show V-shaped accuracy curves and model rankings that challenge filter-based analyses, plus a case study relating benchmark rankings to rankings on six real node-classification datasets.","tokens_in":25364,"tokens_out":9690,"duration_ms":94438,"significance":"If the central claim holds, the paper's exploratory result would be a valuable caution against attributing GNNs' spectral behavior solely to the aggregation filter, and the proposed benchmark would provide a common spectral yardstick for comparing spatial and spectral GNNs. The paper also has concrete strengths: it ships code, uses six real datasets and 14 architectures, reports standard deviations, and includes a case study with Kendall distances. However, the theoretical foundation is currently not solid, and the main empirical claims lack a no-graph control; both are needed to support the benchmark's stated purpose.","major_comments":[{"comment":"The bound max ||e_u - e(v)||_2 ≤ 2(4n/k^2)^(1/n) is vacuous for the graph sizes and class counts used in the benchmark: for n = 1,000 and k = 10 the right-hand side is about 2.0, which exceeds the maximum possible L2 distance (√2) between two nonnegative unit-norm energy distributions. The proof also contains a dimensional error: the diagonal of a hypercube with side length 2/k is 2√n/k, not (4n/k^2)^(1/n), and the step replacing (1/n)^(1/n) by e^(1/e) in Eq. (2) goes in the wrong direction. Therefore the stated theoretical support for the discretization protocol is not established; the bound should be recomputed or the theoretical claim should be withdrawn.","section":"Section 4.2 / Appendix B, Theorem 4.4"},{"comment":"The benchmark protocol lacks a no-graph baseline. The V-shaped accuracy curves are interpreted as evidence about the frequency-capturing capability of GNNs and specifically as revealing 'an inherent limitation of neighborhood aggregation mechanism' (Section 5.2). However, the labels are quantized entries of graph-Laplacian eigenvectors and the input features are fixed node attributes; a plain MLP or logistic-regression model that ignores the adjacency matrix could in principle exhibit the same V-shape if mid-frequency labels are intrinsically harder to predict from the node features. Without reporting accuracy curves and AUAC rankings for such a control, the attribution of the V-shape to neighborhood aggregation is unsupported. Adding an MLP baseline is necessary to validate RQ1 and RQ2.","section":"Sections 4.1 and 5.2"},{"comment":"The theoretical parameter k is not connected to the experimental protocol. Theorems 4.3 and 4.4 concern the number k of equal-length intervals in [-1,1] used to discretize an eigenvector's entries, but the main benchmark uses frequency bins of width 0.1 and the number of classes is only illustrated with an example ('5') in Appendix A.2. The actual k per dataset is not reported, and the theorems do not directly address whether a classifier trained on the discretized labels preserves the frequency-specific incentives of the continuous target. Even a sharp bound on energy deviation of some unit vector with the same NCL matrix would not, by itself, imply that the node classification accuracy curves measure the intended frequency-capturing capability. Please spell out the mapping or provide a direct argument.","section":"Section 4.1 / Appendix A.2"}],"minor_comments":[{"comment":"The notation paragraph says 'we employ Xi and Xi to indicate its i-th row and column' but the two symbols are identical; this is presumably a typo and should be fixed.","section":"Section 2"},{"comment":"There are several grammatical slips, e.g., 'may also significantly affecting' in the abstract; the manuscript should be carefully proofread.","section":"Abstract and Section 1"},{"comment":"The theorem statement has 'Laplacian amtrix' instead of 'Laplacian matrix'; this typo also appears in the proof heading.","section":"Theorem 4.4 / Appendix B"},{"comment":"The text defines Kendall's τ as 'total number of inversions' and then says smaller values indicate larger similarity; this is actually a Kendall distance, not the normalized τ in [-1,1]. Please clarify the definition and report the normalization used.","section":"Section 5.4"},{"comment":"The figure caption and x-axis include 'Texas' but the text introduces only Airport-Brazil, Wisconsin, Cornell, Squirrel, and Chameleon; please reconcile the dataset list.","section":"Figure 4 and text"},{"comment":"The table headers say 'multiplied by 10' but the surrounding text says 'All metrics are multiplied by 100'; these should be made consistent.","section":"Appendix E, Tables 3–5"}],"recommendation":"major_revision","confidential_remarks":"The exploratory study in Section 3 is the strongest part of the paper and may be publishable on its own; the benchmark and its theory need substantial revision. The missing no-graph baseline and the vacuous bound are fixable in principle, but the current form overclaims the theoretical and empirical support. I would encourage the editor to invite a revised version rather than reject outright."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the exploratory experiment is the real contribution; the benchmark is a useful first attempt but its central interpretation is currently unvalidated because there is no graph-agnostic control. The theory is window dressing and should be cut down.\n\nWhat's new and good: the paper demonstrates with energy plots that a standard GCN, trained on low-frequency-only input, can produce high-frequency output, and vice versa. That's a clean, reproducible check on the claim that a GNN isn't just its aggregation filter. The benchmark, with 14 GNNs and six datasets, gives a concrete way to compare models across the spectral axis, and the Normalized AUAC metric is simple enough to adopt. The Kendall-tau case study is a reasonable start at external validation.\n\nWhere it falls down: the V-shaped accuracy curves are interpreted as an 'inherent limitation of neighborhood aggregation,' but the protocol never includes a no-graph classifier. An MLP could show the same V-shape if mid-band labels are simply harder to learn from the raw node features. Until that control is run, the benchmark is measuring something about the synthetic label-generation process, not necessarily about GNN spectral capability. The case study doesn't resolve this because it only checks whether frequency-range rankings transfer to real tasks; it doesn't tell us what drives the curve shape.\n\nThe theoretical section is also not up to the claim. Theorem 4.4 gives a bound that is about 2 for any graph of realistic size, so it says nothing useful about energy-distribution preservation under discretization. The appendix even contains a few 'we consider these outliers' exclusions, which contradict the main text's 'all cases' claim and are not justified.\n\nWho reads this: people working on spectral analysis of GNNs or building GNN benchmarks. It's a serious attempt at an important problem. Send it to reviewers, but they should demand a no-graph baseline, a major rewrite of the theory, and honest handling of outliers. With those revisions, the benchmark could become useful.","headline":"A useful empirical challenge to the filter-only view of GNNs, but the benchmark's claims need a no-graph control before they can be trusted.","tokens_in":25873,"tokens_out":5062,"would_cite":false,"duration_ms":55530,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Filters don't decide what GNNs can learn: a benchmark of 14 models shows graph neural networks can recreate frequency components missing from their input, so spectral analysis of the aggregation filter alone is incomplete.","keywords":["graph neural networks","spectral analysis","frequency components","graph Laplacian","neighborhood aggregation","node classification benchmark","spectral filtering","non-linear layers"],"falsifier":"Replace a GNN's aggregation step with a hard band-stop projection that sets the filter response to exactly zero on one bin of the Laplacian spectrum while keeping the non-linear layers, and run the paper's frequency-incentive protocol. If the output energy in the zeroed bin stays at zero even when the supervision targets that bin, the aggregation filter's frequency response does determine the GNN's spectral behavior in that regime, contradicting the central claim; conversely, if the GNN regenerates the band, the claim survives a strict test.","tokens_in":24864,"feed_emoji":"📊","tokens_out":14156,"duration_ms":135460,"temperature":0.7,"pith_summary":"This paper challenges the prevailing view that a GNN's spectral behavior is determined by the frequency response of its neighborhood aggregation filter. Through a regression experiment where the input signal and the prediction target live in disjoint frequency bands, the paper shows that GCN can still produce outputs aligned with the target band, meaning the network as a whole can regenerate frequency components that are absent from its input. To make this measurable, the paper designs a benchmark: labels are generated by binning graph-Laplacian eigenvectors by eigenvalue and discretizing them, and each of 14 GNNs is scored on its accuracy per frequency bin. The results show V-shaped accuracy curves across models and datasets (lowest and highest frequencies are easiest, the middle band hardest) and rankings that favor models rarely analyzed spectrally, such as SAGE, GCNII, and GATv2. The upshot is that spectral analyses treating the aggregator as the sole filter are incomplete; other modules, notably non-linear layers, materially reshape a GNN's output spectrum.","feed_headline":"Filters don't decide what GNNs can learn, benchmark finds","feed_subtitle":"A 14-model benchmark shows GNNs can recreate frequencies missing from their input, so filter-only analyses fall short.","key_machinery":"The load-bearing object is a pair of spectral quantities built on the normalized graph Laplacian $L = I - D^{-1/2}AD^{-1/2}$. The first is per-bin signal energy: eigenvectors of $L$ are sorted by eigenvalue (frequency), binned, and the mean eigenvector per bin is used as input or target signal, with energy per bin defining the Energy Distribution Field of a label matrix. The second is the evaluation protocol: each bin's continuous values are discretized into class labels, a GNN is trained to predict those labels, and its accuracy per bin traces an accuracy curve across the spectrum, summarized by the Normalized Area Under the Accuracy Curve (AUAC). The theoretical support is an energy distribution function $e(v) = (Av) \\odot (Av)/\\|Av\\|^2$ with orthonormal $A$, shown to be Lipschitz on the unit sphere, plus a bound (Theorem 4.4) that discretizing an eigenvector into $k$ equal intervals shifts its energy distribution by at most $2(4n/k^2)^{1/n}$. This justifies reading classification accuracy on binned-eigenvector labels as a measure of how well a GNN captures the information encoded in that frequency component.","core_discovery":"The paper's central claim: the frequency response of the filter of a neighborhood aggregator does not necessarily determine the behavioral characteristics of its host GNN in the frequency domain. Concretely, the paper constructs node regression tasks in which the input features are the mean eigenvectors from one third of the spectral range (say low) while the prediction target is the mean of eigenvectors from another third (say high), so the target frequencies carry zero input energy. GCN trained under MSE loss still matches the target energy distribution in both directions (low-to-high and high-to-low), and does so on par with FA, a GNN built on a learnable spectral filter. The paper concludes that non-linear layers and other non-filter components let GNNs flexibly shift output energy across the spectrum, and that filter-based spectral analysis alone cannot explain a GNN's strengths and weaknesses. It then formalizes this into a benchmark problem, proposes an accuracy-curve protocol with a Normalized AUAC score, proves an upper bound on the spectral deviation introduced by discretizing continuous eigenvector targets into class labels, and reports the resulting rankings for 14 GNNs on six datasets.","pith_inferences":["If GNNs can regenerate frequencies that the input lacks, then feature design and supervision signal matter as much as filter design; a natural test is whether GNNs trained with middle-band targets on graphs engineered for homophily can be pushed above the observed V-shaped floor, which the paper does not attempt.","The band-specific rankings suggest a falsifiable design rule: a GNN with an explicit mid-frequency bias (e.g., aggregation on a band-passed neighborhood) should beat all 14 baselines on middle-band targets, and the benchmark provides the exact protocol to check this.","The benchmark's validity rests on synthetic eigenvector-derived labels; extending the Kendall-tau case study to a broader suite of real tasks whose labels' dominant band is measured from the Laplacian spectrum would tell whether the rankings transfer beyond the six datasets tested.","The finding that plain GCN matches FA, a model with a learnable filter, on regeneration tasks implies that expensive filter designs may yield their gains not from the filter itself but from the non-linear readout; comparing FA against GCN with matched non-linear layers would separate these contributions."],"forward_implications":["Spectral analyses that read a GNN's behavior off its aggregator's frequency response alone are incomplete; conclusions about low-pass or band-pass behavior must be re-checked on the whole network.","On the benchmark's protocol, all 14 models show V-shaped accuracy curves, so middle-frequency information appears to be an inherent weak spot of neighborhood aggregation, not a fixable filter-design issue.","The measured rankings give practitioners a model-choice rule: pick a GNN by the dominant frequency band of the task labels (e.g., SAGE and Cheb for low bands, GATv2 and GCN for middle bands, Transformer and GPS for high bands), and the six-dataset case study shows these rankings track actual task rankings better than random rankings or original-task rankings.","The theoretical bound guarantees that discretizing continuous eigenvector targets into class labels changes the target's energy distribution by at most $2(4n/k^2)^{1/n}$, so node-classification results from the benchmark are consistent with the node-regression exploratory findings.","Deepening GNNs from two to four layers does not change the shape of the accuracy curves, so conclusions drawn from analyzing stacked aggregation filters across depths need revision."],"supporting_citations":[{"why":"Supplies the low-pass filter view of neighborhood aggregation that the paper's central claim directly contradicts.","marker":"Nt & Maehara, 2019"},{"why":"Defines GCN, the model whose frequency-regeneration behavior the exploratory study demonstrates.","marker":"Kipf & Welling, 2017"},{"why":"Defines SGC, the simplified essentially linear GNN in the benchmark that anchors the contrast with non-linear models.","marker":"Wu et al., 2019"},{"why":"Provides prior evidence that non-linear layers shift GNN output frequency components, the phenomenon the paper generalizes.","marker":"Balcilar et al., 2021b"},{"why":"Defines FA, the learnable-filter GNN whose frequency flexibility plain GCN is shown to match.","marker":"Bo et al., 2021"},{"why":"Supplies the notion of spectral energy used throughout to compare input, target, and output frequency content.","marker":"Yang et al., 2022"},{"why":"Provides four of the six real-world benchmark datasets (CS, Physics, Computers, Photo) used in the evaluation.","marker":"Shchur et al., 2018"},{"why":"Provides the Cora-full and DBLP citation graphs used in the benchmark.","marker":"Bojchevski & Günnemann, 2018"}],"fun_headline_variants":["GNNs can recreate frequencies their filters block, benchmark finds","Filters don't govern what GNNs can learn, spectral benchmark shows","GNNs are more than their filters, new spectral benchmark proves","Spectral benchmark: GNNs flexibly produce frequencies absent in input","Filter frequency response doesn't cap GNN abilities, benchmark finds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise of the evaluation protocol is that predicting labels made from graph-Laplacian eigenvectors in a frequency bin measures how well a GNN captures the information in that frequency component; if real supervised tasks encode their label information differently, the benchmark's rankings and curves may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["GNNs can recreate frequencies their filters block, benchmark finds","Filters don't govern what GNNs can learn, spectral benchmark shows","GNNs are more than their filters, new spectral benchmark proves","Spectral benchmark: GNNs flexibly produce frequencies absent in input","Filter frequency response doesn't cap GNN abilities, benchmark finds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000561,"raw_usage":{"total_tokens":2700,"prompt_tokens":1016,"completion_tokens":1684,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":1593}},"tokens_in":632,"tokens_out":1684,"duration_ms":12900,"temperature":1.0,"reasoning_tokens":1593,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:03:52.241190+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace a GNN's aggregation step with a hard band-stop projection that sets the filter response to exactly zero on one bin of the Laplacian spectrum while keeping the non-linear layers, and run the paper's frequency-incentive protocol. If the output energy in the zeroed bin stays at zero even when the supervision targets that bin, the aggregation filter's frequency response does determine the GNN's spectral behavior in that regime, contradicting the central claim; conversely, if the GNN regenerates the band, the claim survives a strict test.","supporting_citations":[],"review_version":1}