RICA replaces ICA's global generative model with local Riemannian geometry, introducing a disentanglement tensor based on the Hessian of the log-likelihood and Ricci curvature to measure pointwise disentanglement, which recovers sources across manifolds in controlled tests.
super hub Canonical reference
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
Canonical reference. 100% of citing Pith papers cite this work as background.
122
Pith papers citing it
557
external citations · Pith
Background
100% of classified citations
abstract
The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.
hub tools
citation-role summary
background 15
citation-polarity summary
claims ledger
- abstract The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second,
- background 06128, 2018. [11] Tarek R Besold, Artur d'Avila Garcez, Sebastian Bader, Howard Bowman, Pedro Domingos, Pascal Hitzler, Kai-Uwe Levy, Luis C Lamb, et al. Neural-symbolic learning and reasoning: A survey and interpretation.arXiv preprint arXiv:1711.03902, 2017. [12] Stephen Boyd and Lieven Vandenberghe.Convex optimization. Cambridge University Press, 2004. [13] Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veliˇckovi'c. Geometric deep learning: Grids, groups, graphs, geodesics, and gauge
- background This research is partially supported by the National Research Foundation Singapore under the AI Singapore Programme (AISG Award No: AISG2-TC-2023-010-SGIL) and the Singapore Ministry of Education Academic Research Fund Tier 1 (Award No: T1 251RES2207). References [1] William L. Hamilton. Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3):1-159, 2020. [2] Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veliˇckovi'c. Geometric deep lear
- background URL https://arxiv.org/abs/2104.13478. Preprint at https://arxiv.org/abs/2104.13478. [57] Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. InProceedings of the 40th International Conference on Machine Learning, volume 202 ofProceedings of Machine Learning Research, pages 26517-26582, 2023. [58] Rong Tang and Yun Yang. Adaptivity of diffusion models to manifold structures. InInternational Conference on Artificial Intelligence and Statist
- background be the more fundamental object and construct a graph through it, we forgo any mention to permutations of the nodes and simply call the graph circulant if the adjacency matrix we use for its construction is circulant. 2 can be defined in a natural way [3]. Such graphs have seen increased use in the field of machine learning with the rise of Geometric Deep Learning [4], where their inherent rotational invariance is leveraged [5], as well as a means to compress the weight matrices' sizes and expedi
- background InInternational conference on machine learning, pp. 2806-2823. PMLR, 2023. [20] Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Marc Peter Deisenroth. Matérn gaussian processes on Riemannian manifolds. InAdvances in Neural Information Processing Systems, volume 33, 2020. URL https://proceedings.neurips. cc/paper/2020/hash/92bf5e6240737e0326ea59846a83e076-Abstract. html. [21] Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veliˇckovi'c. Geometric deep learning: Grids, group
- background terpreted as the valuef θ(x) of the model. A growing body of literature argues that Quantum Neural Networks have a spectral bias [39-42] which can be manipulated for specific learning tasks [95]. This includes a "hard" spec- tral bias stemming from the embedding of classical data and a potential "soft" spectral bias that regularises the underlying model class [39], as well as a possible spectral bias with respect to the learning dynamics similar to the one observed in classical neural networks [
authors
co-cited works
fields
cs.LG 64 cs.CV 10 quant-ph 9 cs.AI 4 math.DS 3 q-bio.NC 3 stat.ML 3 cs.CL 2 eess.IV 2 math.DG 2roles
background 14polarities
background 14representative citing papers
HodgeCover isolates the harmonic kernel of a simplicial Laplacian on an expert 2-complex to identify irreducible merge cycles and selects experts for aggressive compression, matching or exceeding baselines on open-weight MoE models.
Hypergraph neural networks obey a strict expressivity hierarchy indexed by hypertree width, creating a Width Wall that no fixed-depth model, hidden dimension, or training procedure can cross for wider patterns.
Neural LoFi models deep learning as layer-wise spectral filtering that selects maximal low-degree correlations, yielding a tractable surrogate for hierarchical representation learning beyond the lazy regime.
NLI autonomously discovers a vocabulary of primitive operations and interprets variable-length programs via a neural executor, allowing end-to-end training and gradient-based test-time adaptation that outperforms prior methods on combinatorial generalization tasks.
Characterizes constituents of n-qubit graph quantum ML models and supplies a toolbox enabling integration with classical models, generalization of prior GQML approaches, and classical pre-training.
Introduces structured DRO for learned inverse problem reconstructions with ambiguity sets aligned to the forward operator, yielding explicit dual representations and a worst-case bound that induces Tikhonov regularization on the operator Lipschitz constant.
Observable Matrix Dynamics (OMD) is a new diagnostic framework that uses random matrix theory on distance matrices to distinguish diffusive relaxations from phase-transition-like reorganizations during neural network training.
A coherence law based on the readout-visible aligned coherence rate (a Rayleigh quotient of the noise generator) predicts gradient survival in noisy U(1)-equivariant QNNs, with simulations confirming R²=0.979 and a special channel test showing no loss where predicted.
Constructs G-equivariant ViTs for arbitrary discrete G ≤ O(2), proves H ≤ G implies G-models embed into H-models and single-head equivariant attention realizes all ordinary G-equivariant maps, introduces D6 hexagonal model, and reports preliminary accuracy gains on PatternNet in low-data regimes.
CIPE constructs graph positional encodings from communicability so that self-attention similarities equal the sum of all-path contributions between nodes, yielding 35.5% average gains on seven benchmarks over structure-agnostic Transformers.
Reframing test-time canonicalization as OOD detection allows systematic use of ~20 OOD scores and 9 search algorithms, with distance-based scores plus random search plus local refinement performing best, plus a gated mechanism to preserve in-distribution accuracy.
Data symmetries generically do not induce conserved quantities in NN training for analytic non-polynomial losses, but can for MSE with tensorizable networks.
LH-NeF learns tokenized neural-field representations via a locality-preserving hierarchical encoder, achieving 42× lower memory and 133× larger batches than modality-agnostic meta-learning baselines while matching or exceeding performance on reconstruction and downstream tasks.
An exact algebraic identity plus low-rank SVD and Haar-measure null-space approximation reduce per-point mean curvature cost from O(m^4) to O(k^2 m + k m p^2) with 50-300x speedups and negligible accuracy loss.
Gradient flow on a two-layer network trained to compose finite-group elements provably pushes each neuron to a single irreducible representation with rank-one cross-layer alignment; for Abelian groups it yields a uniformly diversified, Haar-phase majority-vote predictor.
Introduces coherence as a topological constraint on representations and the Coh objective to enforce geometric clustering for interpretability in neural networks.
A representation property is identifiable from the induced predictor iff it is constant on the fibers of the map from admissible (representation, head) pairs to the composite predictor.
On a C_n-symmetric task, wrong-group symmetry priors reduce performance versus no prior (CI excludes zero), augmentation matches equivariant models, and measured symmetry-data exchange rate is 1.28 (wide CIs include zero).
Cellular Sheaf Neural Operators use cell complexes, learned restriction maps, and structure-aware message passing to create discretization-aware neural surrogates that preserve constraints in multiphysics PDEs such as MHD.
TriSearch is an RL framework that optimizes triangulations of polytopes using bistellar flips with a circuit-supported subtriangulation action representation, generalizing zero-shot to larger instances and outperforming prior samplers in 3D and 4D.
Introduces graph-to-image prediction of per-node dynamic stability landscapes in oscillator networks from topology, releases two 10k-graph datasets, and shows GNN-CNN models achieve good accuracy with cross-size generalization.
Introduces rewriting categories to formalize proof equivariance and success invariance, shows LLM provers violate both, and demonstrates test-time aggregation recovers invariance and boosts performance.
Gaussian Sheaf Neural Networks derive a sheaf Laplacian for Gaussian node features on graphs to preserve their geometric structure during message passing.
citing papers explorer
-
Disentanglement Beyond Generative Models with Riemannian ICA
RICA replaces ICA's global generative model with local Riemannian geometry, introducing a disentanglement tensor based on the Hessian of the log-likelihood and Ricci curvature to measure pointwise disentanglement, which recovers sources across manifolds in controlled tests.
-
HodgeCover: Higher-Order Topological Coverage Drives Compression of Sparse Mixture-of-Experts
HodgeCover isolates the harmonic kernel of a simplicial Laplacian on an expert 2-complex to identify irreducible merge cycles and selects experts for aggressive compression, matching or exceeding baselines on open-weight MoE models.
-
The WidthWall: A Strict Expressivity Hierarchy for Hypergraph Neural Networks
Hypergraph neural networks obey a strict expressivity hierarchy indexed by hypertree width, creating a Width Wall that no fixed-depth model, hidden dimension, or training procedure can cross for wider patterns.
-
Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
Neural LoFi models deep learning as layer-wise spectral filtering that selects maximal low-degree correlations, yielding a tractable surrogate for hierarchical representation learning beyond the lazy regime.
-
Gradient-Based Program Synthesis with Neurally Interpreted Languages
NLI autonomously discovers a vocabulary of primitive operations and interprets variable-length programs via a neural executor, allowing end-to-end training and gradient-based test-time adaptation that outperforms prior methods on combinatorial generalization tasks.
-
Quantum machine learning models for graphs
Characterizes constituents of n-qubit graph quantum ML models and supplies a toolbox enabling integration with classical models, generalization of prior GQML approaches, and classical pre-training.
-
A Distributionally Robust Framework for Learned Reconstructions in Inverse Problems
Introduces structured DRO for learned inverse problem reconstructions with ambiguity sets aligned to the forward operator, yielding explicit dual representations and a worst-case bound that induces Tikhonov regularization on the operator Lipschitz constant.
-
Learning as Observable Matrix Dynamics: Diffusive Relaxations versus Phase Transitions
Observable Matrix Dynamics (OMD) is a new diagnostic framework that uses random matrix theory on distance matrices to distinguish diffusive relaxations from phase-transition-like reorganizations during neural network training.
-
A Coherence Law for Trainability in Noisy Equivariant Quantum Neural Networks
A coherence law based on the readout-visible aligned coherence rate (a Rayleigh quotient of the noise generator) predicts gradient survival in noisy U(1)-equivariant QNNs, with simulations confirming R²=0.979 and a special channel test showing no loss where predicted.
-
A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of $\mathrm{O}(2)$
Constructs G-equivariant ViTs for arbitrary discrete G ≤ O(2), proves H ≤ G implies G-models embed into H-models and single-head equivariant attention realizes all ordinary G-equivariant maps, introduces D6 hexagonal model, and reports preliminary accuracy gains on PatternNet in low-data regimes.
-
Communicability-Inspired Positional Encoding (CIPE)
CIPE constructs graph positional encodings from communicability so that self-attention similarities equal the sum of all-path contributions between nodes, yielding 35.5% average gains on seven benchmarks over structure-agnostic Transformers.
-
Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring
Reframing test-time canonicalization as OOD detection allows systematic use of ~20 OOD scores and 9 search algorithms, with distance-based scores plus random search plus local refinement performing best, plus a gated mechanism to preserve in-distribution accuracy.
-
Conservation Laws from Data Symmetry in Neural Networks
Data symmetries generically do not induce conserved quantities in NN training for analytic non-polynomial losses, but can for MSE with tensorizable networks.
-
Neural Field Tokenizations with Hierarchy and Spatial Locality Priors
LH-NeF learns tokenized neural-field representations via a locality-preserving hierarchical encoder, achieving 42× lower memory and 133× larger batches than modality-agnostic meta-learning baselines while matching or exceeding performance on reconstruction and downstream tasks.
-
Efficient Mean Curvature Computation on High-Dimensional Data Manifolds
An exact algebraic identity plus low-rank SVD and Haar-measure null-space approximation reduce per-point mean curvature cost from O(m^4) to O(k^2 m + k m p^2) with 50-300x speedups and negligible accuracy loss.
-
Neural Networks Provably Learn Spectral Representations for Group Composition
Gradient flow on a two-layer network trained to compose finite-group elements provably pushes each neuron to a single irreducible representation with rank-one cross-layer alignment; for Abelian groups it yields a uniformly diversified, Haar-phase majority-vote predictor.
-
Learning Coherent Representations: A Topological Approach to Interpretability
Introduces coherence as a topological constraint on representations and the Coh objective to enforce geometric clustering for interpretability in neural networks.
-
A Fiber Criterion for Representation Identifiability in Supervised Learning
A representation property is identifiable from the induced predictor iff it is constant on the fibers of the map from admissible (representation, head) pairs to the composite predictor.
-
Measuring the Symmetry--Data Exchange Rate
On a C_n-symmetric task, wrong-group symmetry priors reduce performance versus no prior (CI excludes zero), augmentation matches equivariant models, and measured symmetry-data exchange rate is 1.28 (wide CIs include zero).
-
Cellular Sheaf Neural Operators for Structure-Preserving Surrogate Modeling of Constrained PDEs
Cellular Sheaf Neural Operators use cell complexes, learned restriction maps, and structure-aware message passing to create discretization-aware neural surrogates that preserve constraints in multiphysics PDEs such as MHD.
-
TriSearch: Learning to Optimize Triangulations via Bistellar Flips
TriSearch is an RL framework that optimizes triangulations of polytopes using bistellar flips with a circuit-supported subtriangulation action representation, generalizing zero-shot to larger instances and outperforming prior samplers in 3D and 4D.
-
Learning Dynamic Stability Landscapes in Synchronization Networks
Introduces graph-to-image prediction of per-node dynamic stability landscapes in oscillator networks from topology, releases two 10k-graph datasets, and shows GNN-CNN models achieve good accuracy with cross-size generalization.
-
What are the Right Symmetries for Formal Theorem Proving?
Introduces rewriting categories to formalize proof equivariance and success invariance, shows LLM provers violate both, and demonstrates test-time aggregation recovers invariance and boosts performance.
-
Gaussian Sheaf Neural Networks
Gaussian Sheaf Neural Networks derive a sheaf Laplacian for Gaussian node features on graphs to preserve their geometric structure during message passing.
-
Physics-Aligned Canonical Equivariant Fourier Neural Operator under Symmetry-Induced Shifts
PACE-FNO reduces OOD relative error by up to 12x versus FNO with symmetry augmentation on Burgers, shallow-water, and Navier-Stokes equations by jointly training a frame estimator and operator under bounded symmetry perturbations.
-
Discretizing Group-Convolutional Neural Networks for 3D Geometry in Feature Space
Feature-space sampling in GCNNs preserves 3D classification accuracy with coarse discretization, enabling precomputation and faster training of equivariant models.
-
Matrix-Space Reinforcement Learning for Reusing Local Transition Geometry
MSRL represents trajectory segments as PSD matrices to prove additive composition properties and bootstrap value functions for better transfer, reaching 0.73 AUC versus 0.57-0.65 baselines.
-
TokaMind for Power Grid: Cross-Domain Transfer from Fusion Plasma
TokaMind, pre-trained on MAST tokamak data, transfers to power grid PMU data for severe event classification with F1 0.837, where difficulty depends on grid topology and CSD indicators boost early-warning performance over CNN baselines.
-
Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity
Every fixed finite feedforward neural network definable in an o-minimal structure has finite sample complexity in the agnostic PAC setting.
-
Operator-Guided Invariance Learning for Continuous Reinforcement Learning
VPSD-RL discovers exact and approximate value-preserving Lie-group operators in continuous RL to stabilize learning via transition augmentation and consistency regularization.
-
Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves
HilbNets define convolutions via Hilbert bundle connection Laplacians, prove that sampled Hilbert cellular sheaf Laplacians converge to the continuous operator, and show that discretized networks are consistent and transferable across samplings.
-
Cardiac Mesh Flow: One-Step Generation of 3D+t Cardiac Four-Chamber Meshes via Flow Matching
Cardiac Mesh Flow generates 3D+t four-chamber cardiac meshes with anatomical correspondence and volume conditioning via one-step flow matching on multi-scale deformation fields.
-
Data-driven discovery of polynomial ODEs with provably bounded solutions
SILAS jointly optimizes polynomial ODE vector fields and polynomial Lyapunov functions from data to produce models with provably bounded trajectories via compact absorbing sets.
-
Complex-Valued GNNs for Distributed Basis-Invariant Control of Planar Systems
Complex-valued GNNs using phase-equivariant activations achieve global basis invariance for distributed planar control, outperforming real-valued baselines in data efficiency, tracking, and generalization on flocking.
-
Exact Verification of Graph Neural Networks with Incremental Constraint Solving
Develops an exact verification method for GNNs supporting sum, max and mean aggregations via incremental constraint solving with bound tightening for adversarial robustness on node and graph classification tasks.
-
HSG-12M: A Large-Scale Benchmark of Spatial Multigraphs from the Energy Spectra of Non-Hermitian Crystals
HSG-12M is a large dataset of spatial multigraphs derived from non-Hermitian crystal energy spectra via the Poly2Graph pipeline, positioned as the first large-scale benchmark of this graph type.
-
Massive Activations in Large Language Models
Massive activations are constant large values in LLMs that function as indispensable bias terms and concentrate attention probabilities on specific tokens.
-
Stability of Flow Models for Graph Signals
GNN-parametrized continuous normalizing flows for graph signals are permutation equivariant and satisfy Wasserstein stability bounds under relative graph perturbations, motivating a Lipschitz-regularized training strategy.
-
When Do Geometric Algebra Layers Beat Scalarization? A Controlled Study on SO(3)-Equivariant Vector Laws
Cl(3,0) geometric algebra layers beat scalarization only on nested group-element compositions in low data; on single-stage vector laws scalarization matches or wins at far lower cost.
-
Spin-Weighted Spherical Harmonics Enable Complete and Scalable $\mathrm{E}(3)$-Equivariant Networks
Spin-weighted spherical harmonics recover the antisymmetric paths that scalar Gaunt tensor products drop, yielding complete E(3) tensor products at Gaunt-like asymptotic cost.
-
Group-invariant Coresets for Data-efficient Active Learning
GRINCO performs acquisition in the quotient space induced by a transformation group using invariant embeddings or canonical representatives, pairs it with orbit-averaged loss, derives a generalization bound, and reports better orbit coverage and label efficiency than standard coresets on synthetic a
-
From Scene-Centric to Observer-Centric: Modeling Observer-Aware Relations for 3D Scene Graph Generation
Decoupling yaw-invariant from direction-sensitive predicates in separate relation branches makes 3D scene graph prediction robust to 90°/180°/270° viewpoint rotations on 3DSSG without rotation augmentation.
-
Scalable Message-Passing Quantum Graph Neural Networks in the Weisfeiler-Leman Hierarchy
The work constructs a permutation-equivariant quantum GNN that implements message passing at selectable Weisfeiler-Leman levels, supports pre-training on small graphs, and demonstrates readout scalability with simulations up to 56 qubits on synthetic, molecular, and TSP datasets.
-
Enhancing LLMs for Graph Tasks via Graph-aware LoRA Generation
GaRA generates task-specific LoRA weight updates conditioned on graph structures to enable better whole-graph encoding in LLMs for zero-shot graph learning.
-
Symmetry Alone Is Not an Ansatz: Task-Aligned Interactions in Equivariant Quantum Circuits
In Tic-Tac-Toe, adding trainable gates on the eight winning triples improves an equivariant quantum classifier more than strengthening the imposed symmetry.
-
Algebraic Networks and Architectural Degenerations
Develops algebraic geometry tools for monomial neural networks and proves the singular locus of neurovarieties is contained in the architectural degeneracy locus for fully connected networks with non-increasing widths and scalar output under layerwise regularity assumptions.
-
Charting the emergent low-dimensional manifold of quantum materials
Unsupervised manifold learning on ICSD data reveals a low-dimensional embedding that segregates superconductors and predicts critical temperatures across families.
-
MViewRouter: Internalizing Geometric Equivariance via Multi-view Alternating Attention for Combinatorial Routing
MViewRouter internalizes D4 geometric equivariance for routing via Multi-view Alternating Attention and Collective Policy Gradient Aggregation, yielding competitive solutions and strong generalization on TSP/CVRP benchmarks.
-
LAST: Bridging Vision-Language and Action Manifolds via Gromov-Wasserstein Alignment
LAST linearizes action manifolds with Lie-algebraic mapping and discretizes them into approximately isotropic charts to align with VL semantic geometry via Gromov-Wasserstein distance.
-
Sampling Triangulations and Calabi-Yau Threefolds with Autoregressive GNNs
Introduces dualGNN, an autoregressive message-passing GNN using signed circuits to sample uniform fine regular triangulations of lattice polytopes, applied to Calabi-Yau threefolds at h^{1,1}=86 and 128.