Under a denoising objective, linear attention is suboptimal; Graph Convolutional Attention matches idealized spectral attention on SBMs and improves graph denoising and diffusion in proportion to spectral diversity.
Specformer: Spectral graph neural networks meet transformers.arXiv preprint arXiv:2303.01028
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
Spectral graph neural networks (GNNs) learn graph representations via spectral-domain graph convolutions. However, most existing spectral graph filters are scalar-to-scalar functions, i.e., mapping a single eigenvalue to a single filtered value, thus ignoring the global pattern of the spectrum. Furthermore, these filters are often constructed based on some fixed-order polynomials, which have limited expressiveness and flexibility. To tackle these issues, we introduce Specformer, which effectively encodes the set of all eigenvalues and performs self-attention in the spectral domain, leading to a learnable set-to-set spectral filter. We also design a decoder with learnable bases to enable non-local graph convolution. Importantly, Specformer is equivariant to permutation. By stacking multiple Specformer layers, one can build a powerful spectral GNN. On synthetic datasets, we show that our Specformer can better recover ground-truth spectral filters than other spectral GNNs. Extensive experiments of both node-level and graph-level tasks on real-world graph datasets show that our Specformer outperforms state-of-the-art GNNs and learns meaningful spectrum patterns. Code and data are available at https://github.com/bdy9527/Specformer.
fields
cs.LG 7representative citing papers
Crys-JEPA introduces a joint embedding predictive architecture that creates an energy-aware latent space, enabling embedding-based stability screening and a refinement pipeline that yields up to 72.7% gains on the V.S.U.N. metric for crystal generation.
SPECTRA improves molecular property regression on underrepresented targets via spectral graph generation with rarity-aware budgeting and Laplacian interpolation, paired with edge-aware Chebyshev GNNs, yielding competitive benchmark performance at lower compute cost.
S³GNN mitigates oversquashing in message-passing networks via lightweight global mixing without strong prior assumptions, yielding up to 10x error reduction and 50% fewer parameters across multiple domains.
Contrastive FUSE learns node embeddings from partial pairwise supervision and structural signals alone by optimizing a spectral contrastive objective with a lightweight modularity approximation, yielding competitive performance and runtime gains on citation and co-purchase graphs.
VQ-VAE concept learning enables controllable recombination of crystal motifs to generate structures with reported gains in validity-stability-uniqueness-novelty metrics on MP-20 and Alex-MP-20.
FC-GSSL is a graph self-supervised method that corrupts nodes/edges with high low-frequency contribution and reconstructs low-frequency/general targets, improving node and graph prediction on most tested benchmarks.
citing papers explorer
-
Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion
Under a denoising objective, linear attention is suboptimal; Graph Convolutional Attention matches idealized spectral attention on SBMs and improves graph denoising and diffusion in proportion to spectral diversity.
-
Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinement
Crys-JEPA introduces a joint embedding predictive architecture that creates an energy-aware latent space, enabling embedding-based stability screening and a refinement pipeline that yields up to 72.7% gains on the V.S.U.N. metric for crystal generation.
-
SPECTRA: Spectral Domain-Aware Graph Generation for Imbalanced Molecular Property Regression
SPECTRA improves molecular property regression on underrepresented targets via spectral graph generation with rarity-aware budgeting and Laplacian interpolation, paired with edge-aware Chebyshev GNNs, yielding competitive benchmark performance at lower compute cost.
-
S$^3$GNN: Efficient Global Mixing and Local Message Passing for Long-Range Graph Learning
S³GNN mitigates oversquashing in message-passing networks via lightweight global mixing without strong prior assumptions, yielding up to 10x error reduction and 50% fewer parameters across multiple domains.
-
Fast and Featureless Node Representation Learning with Partial Pairwise Supervision
Contrastive FUSE learns node embeddings from partial pairwise supervision and structural signals alone by optimizing a spectral contrastive objective with a lightweight modularity approximation, yielding competitive performance and runtime gains on citation and co-purchase graphs.
-
Composable Crystals: Controllable Materials Discovery via Concept Learning
VQ-VAE concept learning enables controllable recombination of crystal motifs to generate structures with reported gains in validity-stability-uniqueness-novelty metrics on MP-20 and Alex-MP-20.
-
Frequency-Corrupt Based Graph Self-Supervised Learning
FC-GSSL is a graph self-supervised method that corrupts nodes/edges with high low-frequency contribution and reconstructs low-frequency/general targets, improving node and graph prediction on most tested benchmarks.