Hypernetwork generates model parameters from one perturbed low-dimensional private dataset embedding, yielding higher utility than DP-SGD under fixed privacy budget in synthetic theory and lower FID in LoRA diffusion fine-tuning.
Neural network diffusion
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
Diffusion models have achieved remarkable success in image and video generation. In this work, we demonstrate that diffusion models can also \textit{generate high-performing neural network parameters}. Our approach is simple, utilizing an autoencoder and a diffusion model. The autoencoder extracts latent representations of a subset of the trained neural network parameters. Next, a diffusion model is trained to synthesize these latent representations from random noise. This model then generates new representations, which are passed through the autoencoder's decoder to produce new subsets of high-performing network parameters. Across various architectures and datasets, our approach consistently generates models with comparable or improved performance over trained networks, with minimal additional cost. Notably, we empirically find that the generated models are not memorizing the trained ones. Our results encourage more exploration into the versatile use of diffusion models. Our code is available \href{https://github.com/NUS-HPC-AI-Lab/Neural-Network-Diffusion}{here}.
representative citing papers
TFlow enables multi-agent LLMs to collaborate via transient low-rank LoRA perturbations derived from sender activations, yielding up to 8.5 accuracy gains and 83% token reduction versus text-based baselines on Qwen3-4B models.
DynaDiff uses weight-graph diffusion with a functional consistency loss and dynamics-informed prompting to generate adapted predictors, reporting 10.78% average accuracy gains over baselines while amortizing adaptation cost offline.
A learned local rule (Weight Transformer) iteratively self-organizes task-network weights from local graph neighborhoods and generalizes across unseen MLP, CNN, and ResNet architectures up to ~2M parameters.
KSDiff generates convolutional kernels in kernel space using low-rank core tensor and factor generators with multi-head attention for fast, high-quality pansharpening.
A bidirectional optimization method using parameterized transformations enables near-zero loss barriers for linear mode connectivity in medium-scale language models and small barriers in billion-parameter transformers.
Diffusion models are evolutionary algorithms via a denoising-evolution equivalence, yielding Diffusion Evolution that outperforms mainstream EAs on multi-optima tasks.
citing papers explorer
-
Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork
Hypernetwork generates model parameters from one perturbed low-dimensional private dataset embedding, yielding higher utility than DP-SGD under fixed privacy budget in synthetic theory and lower FID in LoRA diffusion fine-tuning.
-
Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights
TFlow enables multi-agent LLMs to collaborate via transient low-rank LoRA perturbations derived from sender activations, yielding up to 8.5 accuracy gains and 83% token reduction versus text-based baselines on Qwen3-4B models.
-
Generative Adaptation of Dynamics to Environmental Shifts via Weight-space Diffusion
DynaDiff uses weight-graph diffusion with a functional consistency loss and dynamics-informed prompting to generate adapted predictors, reporting 10.78% average accuracy gains over baselines while amortizing adaptation cost offline.
-
Architecture Generalization with MetaNCA
A learned local rule (Weight Transformer) iteratively self-organizes task-network weights from local graph neighborhoods and generalizes across unseen MLP, CNN, and ResNet architectures up to ~2M parameters.
-
Fast Kernel-Space Diffusion for Remote Sensing Pansharpening
KSDiff generates convolutional kernels in kernel space using low-rank core tensor and factor generators with multi-head attention for fast, high-quality pansharpening.
-
Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers
A bidirectional optimization method using parameterized transformations enables near-zero loss barriers for linear mode connectivity in medium-scale language models and small barriers in billion-parameter transformers.
-
Diffusion Models are Evolutionary Algorithms
Diffusion models are evolutionary algorithms via a denoising-evolution equivalence, yielding Diffusion Evolution that outperforms mainstream EAs on multi-optima tasks.