REVIEW 18 cited by
From data to functa: Your data point is a function and you can treat it like one
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
It is common practice in deep learning to represent a measurement of the world on a discrete grid, e.g. a 2D grid of pixels. However, the underlying signal represented by these measurements is often continuous, e.g. the scene depicted in an image. A powerful continuous alternative is then to represent these measurements using an implicit neural representation, a neural function trained to output the appropriate measurement value for any input spatial location. In this paper, we take this idea to its next level: what would it take to perform deep learning on these functions instead, treating them as data? In this context we refer to the data as functa, and propose a framework for deep learning on functa. This view presents a number of challenges around efficient conversion from data to functa, compact representation of functa, and effectively solving downstream tasks on functa. We outline a recipe to overcome these challenges and apply it to a wide range of data modalities including images, 3D shapes, neural radiance fields (NeRF) and data on manifolds. We demonstrate that this approach has various compelling properties across data modalities, in particular on the canonical tasks of generative modeling, data imputation, novel view synthesis and classification. Code: https://github.com/deepmind/functa
Forward citations
Cited by 18 Pith papers
-
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
Permutation-equivariant weight-space networks are all equally expressive, and universality holds when hidden-layer biases are pairwise distinct.
-
FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fields
MDIR detects LLM weight homology from embedding matrices alone using polar decomposition and permutation matching, achieving perfect AUC and accuracy on LeaFBench and reconstructing layer-level transformations.
-
IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos
IM-Portrait generates 3D-aware talking head videos by directly diffusing Multiplane Images, trained on monocular video without multi-view data.
-
Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights
ProbeLog represents each classifier output by its responses to fixed probe images and uses CLIP to answer text queries, achieving 43.8% top-1 accuracy when searching 1,500 ImageNet-trained models for a concept.
-
A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervision
A 3D Gaussian Splat diffusion model trained with only 2D image supervision, using deterministic reconstruction models as noisy teachers, improves single-image 3D reconstruction over those teachers.
-
Topology-Preserving Meshing of Implicit Scalar Fields via Monotonicity Constraints
A refinement algorithm enforces monotonic mesh edges so that piecewise-linear approximations of implicit 2D scalar fields preserve the critical points of the underlying Morse function.
-
Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling
Picasso produces multi-object scene reconstructions that are both geometrically accurate and physically plausible by using physics-constrained rejection sampling over an inferred contact graph, outperforming prior met...
-
Seeing the Many: Exploring Parameter Distributions Conditioned on Features in Surrogates
A Bayesian system that samples and visualizes the distribution of simulation parameters consistent with a user-specified output feature, using a density prior and Hamiltonian Monte Carlo.
-
VidFuncta: Towards Generalizable Neural Representations for Ultrasound Videos
VidFuncta encodes ultrasound videos into static and time-varying latent vectors, improving reconstruction over 2D and 3D baselines while enabling efficient downstream analysis.
-
Deep Active Inference Agents for Delayed and Long-Horizon Environments
A policy-conditional world model trained under active inference enables single-lookahead planning over hundreds of steps and beats a DQN baseline on energy-efficient control of parallel machines.
-
Equivariant Eikonal Neural Networks: Grid-Free, Scalable Travel-Time Prediction on Homogeneous Spaces
E-NES uses Lie-group point-cloud conditioning and equivariant neural fields to make grid-free eikonal travel-time prediction steerable under rotations and translations, with complete invariant features and competitive...
-
Towards scalable surrogate models based on Neural Fields for large scale aerodynamic simulations
MARIO, a modulated conditional neural field with learned SDF geometry codes, predicts RANS flow fields and surface pressures with reported order-of-magnitude accuracy gains on the AirfRANS scarce task and strong resul...
-
Topology Guidance: Controlling the Outputs of Generative Models via Vector Field Topology
Diffusion model sampling guided by SIREN Jacobian signals can place user-specified critical points at chosen locations in generated vector fields while keeping outputs close to the training distribution.
-
Geometric Neural Process Fields
Geometric Neural Process Fields use Gaussian geometric bases and hierarchical latent variables to improve neural process generalization to 1D, 2D, and 3D signals.
-
Scalable and High-Quality Neural Implicit Representation for 3D Reconstruction
A scene is reconstructed as a graph of overlapping local neural SDFs that are registered and blended, improving detail and enabling large-scale reconstruction.
-
Level-Set Parameters: Novel Representation for 3D Shape Analysis
Aligned SDF network weights, conditioned on pose by a hypernetwork, act as a continuous 3D shape representation that performs well on classification, retrieval, and pose estimation.
-
PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling
A Fourier-based weight modulation for shared INR networks improves reconstruction of high-frequency PDE fields and enables bidirectional inference between paired solution spaces.
-
MINR: Implicit Neural Representations with Masked Image Modelling
A hybrid of implicit neural representations and masked image modeling, called MINR, reconstructs masked image patches better than MAE in the reported in-domain and out-of-distribution tests with fewer parameters.
Discussion (0). Continue with ORCID to comment.