AGAN is the first neural architecture search method for GANs that discovers architectures outperforming state-of-the-art on CIFAR-10 unsupervised image generation and competitive on supervised tasks.
hub
DARTS: Differentiable Architecture Search
44 Pith papers cite this work. Polarity classification is still indexing.
44
Pith papers citing it
abstract
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent. Extensive experiments on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in discovering high-performance convolutional architectures for image classification and recurrent architectures for language modeling, while being orders of magnitude faster than state-of-the-art non-differentiable techniques. Our implementation has been made publicly available to facilitate further research on efficient architecture search algorithms.
hub tools
citation-role summary
background 3
citation-polarity summary
roles
background 3polarities
background 3representative citing papers
Hypernetworks distill modular reservoir connectivity via a genomic bottleneck to generate sparse recurrent networks solving difficult temporal tasks with minimal training and maintained robustness.
Introduces the 1GC-7RC benchmark to evaluate AI coding agents on seven diverse ML tasks under single-GPU time and access constraints.
PINNs optimize Dirac operators to satisfy the Ginsparg-Wilson relation, reproducing overlap fermions and autonomously recovering both the standard and a Fujikawa-type generalized GW relation via polynomial ansatz search.
SITE applies soft gradient-based head selection to inject ICL-derived task embeddings, outperforming prior embedding adaptation and few-shot ICL across generation, reasoning, and NLU tasks on 12 LLMs from 4B to 70B parameters.
Switchable Normalization learns per-layer weights to combine channel, layer, and minibatch normalizers, claiming robustness to batch size and better results than fixed normalizers on ImageNet, COCO, CityScapes, ADE20K, MegaFace, and Kinetics.
NetTailor adapts CNN architecture for new tasks by assembling pre-trained universal blocks with task-specific layers, trained via activation mimicry and complexity penalties to match accuracy while reducing size for simpler tasks.
CamoNAS applies neural architecture search with an RGB-frequency dual-stream design to reach state-of-the-art results on four camouflaged object detection benchmarks.
Formalizes Root-Finding Bilevel Optimization (RF-BO) and introduces a Jacobian-free TTSA algorithm that structurally avoids variance amplification, with convergence guarantees and reported gains over squared-residual baselines in SimCLR, ODE control, RL entropy tuning, and generative modeling.
GIF improves sample efficiency and convergence in high-dimensional HPO by using small-sample importance estimates to prioritize trials on high-impact hyperparameters.
A monotone co-design framework for neural network processors that treats uncertainty via Confidence as a tunable resource and allows modular block refinement without structural changes.
DREAM-S combines neural architecture search, target-aware supernet training, and attention-entropy-guided distillation to accelerate speculative decoding in VLMs, reporting up to 3.85x speedup over standard methods.
PACE coordinates low-risk prompt evolution with validated higher-risk control-logic updates to improve frozen SLM agents on benchmarks without model retraining.
AutoMCU uses feasibility-first LLM multi-agent coordination to automate MCU-constrained neural network design, delivering competitive accuracy on CIFAR-10/100 in 1-2 hours versus hundreds of GPU hours for prior HW-NAS methods.
PEML co-optimizes continuous prompts and low-rank adaptations to deliver up to 6.67% average accuracy gains over existing multi-task PEFT methods on GLUE, SuperGLUE, and other benchmarks.
CHAL is a multi-agent dialectic system that performs structured belief optimization over defeasible domains using Bayesian-inspired graph representations and configurable meta-cognitive value system hyperparameters.
Hybrid-LoRA selectively full fine-tunes modules with high sensitivity to low-rank adaptation using a novel score and applies LoRA elsewhere, matching full fine-tuning at 10% budget and outperforming PEFT baselines by up to 5.65%.
SURGE proposes a dual-path gradient compensator and adaptive gradient scaler to mitigate gradient mismatch in binary neural network training via auxiliary backpropagation.
Introduces a novel search direction enabling sublinear stochastic bilevel regret guarantees for first- and zeroth-order online bilevel optimization algorithms without relying on window smoothing.
Progressive widening MCTS with sampling action space automates quantum circuit design, cutting evaluations 10-100x and CNOT gates up to 3x versus prior MCTS on chemistry and linear-equation tasks.
LPS uses a second-order neural network to learn an end-to-end metric for second-order parameter similarity and introduces the ModelSet500 benchmark with 500 trained models.
Uses differentiable NAS with temporal segments and pseudo-3D operators to discover a video action recognition network that outperforms hand-designed models on UCF101 with ~1% of the parameters when trained from scratch.
FLAME models layer-wise overlapping parallelism and asynchronous CPU-GPU pipeline bubbles to estimate inference latency across frequencies with sparse profiling and low error for DNNs and SLMs.
A eight-agent system automatically reproduces and improves SOTA models from top AI papers, reporting 105 surpassing variants on a curated 125-paper set.
citing papers explorer
-
AGAN: Towards Automated Design of Generative Adversarial Networks
AGAN is the first neural architecture search method for GANs that discovers architectures outperforming state-of-the-art on CIFAR-10 unsupervised image generation and competitive on supervised tasks.
-
Distilling a Modular Reservoir Through a Genomic Bottleneck
Hypernetworks distill modular reservoir connectivity via a genomic bottleneck to generate sparse recurrent networks solving difficult temporal tasks with minimal training and maintained robustness.
-
1GC-7RC: One Graphic Card -- Seven Research Challenges! How Good Are AI Agents at Doing Your Job?
Introduces the 1GC-7RC benchmark to evaluate AI coding agents on seven diverse ML tasks under single-GPU time and access constraints.
-
Lattice fermion formulation via Physics-Informed Neural Networks: Ginsparg-Wilson relation and Overlap fermions
PINNs optimize Dirac operators to satisfy the Ginsparg-Wilson relation, reproducing overlap fermions and autonomously recovering both the standard and a Fujikawa-type generalized GW relation via polynomial ansatz search.
-
Soft Head Selection for Injecting ICL-Derived Task Embeddings
SITE applies soft gradient-based head selection to inject ICL-derived task embeddings, outperforming prior embedding adaptation and few-shot ICL across generation, reasoning, and NLU tasks on 12 LLMs from 4B to 70B parameters.
-
Switchable Normalization for Learning-to-Normalize Deep Representation
Switchable Normalization learns per-layer weights to combine channel, layer, and minibatch normalizers, claiming robustness to batch size and better results than fixed normalizers on ImageNet, COCO, CityScapes, ADE20K, MegaFace, and Kinetics.
-
NetTailor: Tuning the Architecture, Not Just the Weights
NetTailor adapts CNN architecture for new tasks by assembling pre-trained universal blocks with task-specific layers, trained via activation mimicry and complexity penalties to match accuracy while reducing size for simpler tasks.
-
CamoNAS: Neural Architecture Search for Enhanced Camouflaged Object Detection
CamoNAS applies neural architecture search with an RGB-frequency dual-stream design to reach state-of-the-art results on four camouflaged object detection benchmarks.
-
Escaping the Variance Trap: Jacobian-Free Dynamics for Root-Finding Bilevel Optimization
Formalizes Root-Finding Bilevel Optimization (RF-BO) and introduces a Jacobian-free TTSA algorithm that structurally avoids variance amplification, with convergence guarantees and reported gains over squared-residual baselines in SimCLR, ODE control, RL entropy tuning, and generative modeling.
-
Importance-Aware Scheduling for High-Dimensional Hyperparameter Optimization
GIF improves sample efficiency and convergence in high-dimensional HPO by using small-sample importance estimates to prioritize trials on high-impact hyperparameters.
-
Uncertainty-Aware End-to-End Co-Design of Neural Network Processors: From Training and Mapping to Fabrication
A monotone co-design framework for neural network processors that treats uncertainty via Confidence as a tunable resource and allows modular block refinement without structural changes.
-
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation
DREAM-S combines neural architecture search, target-aware supernet training, and attention-entropy-guided distillation to accelerate speculative decoding in VLMs, reporting up to 3.85x speedup over standard methods.
-
PACE: Two-Timescale Self-Evolution for Small Language Model Agents
PACE coordinates low-risk prompt evolution with validated higher-risk control-logic updates to improve frozen SLM agents on benchmarks without model retraining.
-
AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systems
AutoMCU uses feasibility-first LLM multi-agent coordination to automate MCU-constrained neural network design, delivering competitive accuracy on CIFAR-10/100 in 1-2 hours versus hundreds of GPU hours for prior HW-NAS methods.
-
PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts
PEML co-optimizes continuous prompts and low-rank adaptations to deliver up to 6.67% average accuracy gains over existing multi-task PEFT methods on GLUE, SuperGLUE, and other benchmarks.
-
CHAL: Council of Hierarchical Agentic Language
CHAL is a multi-agent dialectic system that performs structured belief optimization over defeasible domains using Bayesian-inspired graph representations and configurable meta-cognitive value system hyperparameters.
-
Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
Hybrid-LoRA selectively full fine-tunes modules with high sensitivity to low-rank adaptation using a novel score and applies LoRA elsewhere, matching full fine-tuning at 10% budget and outperforming PEFT baselines by up to 5.65%.
-
SURGE: Surrogate Gradient Adaptation in Binary Neural Networks
SURGE proposes a dual-path gradient compensator and adaptive gradient scaler to mitigate gradient mismatch in binary neural network training via auxiliary backpropagation.
-
Stochastic Regret Guarantees for Online Zeroth- and First-Order Bilevel Optimization
Introduces a novel search direction enabling sublinear stochastic bilevel regret guarantees for first- and zeroth-order online bilevel optimization algorithms without relying on window smoothing.
-
Quantum Circuit Design using a Progressive Widening Enhanced Monte Carlo Tree Search
Progressive widening MCTS with sampling action space automates quantum circuit design, cutting evaluations 10-100x and CNOT gates up to 3x versus prior MCTS on chemistry and linear-equation tasks.
-
Learnable Parameter Similarity
LPS uses a second-order neural network to learn an end-to-end metric for second-order parameter similarity and introduces the ModelSet500 benchmark with 500 trained models.
-
Video Action Recognition Via Neural Architecture Searching
Uses differentiable NAS with temporal segments and pseudo-3D operators to discover a video action recognition network that outperforms hand-designed models on UCF101 with ~1% of the parameters when trained from scratch.
-
Taming Asynchronous CPU-GPU Coupling for Frequency-aware Latency Estimation on Mobile Edge
FLAME models layer-wise overlapping parallelism and asynchronous CPU-GPU pipeline bubbles to estimate inference latency across frequencies with sparse profiling and low error for DNNs and SLMs.
-
AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery
A eight-agent system automatically reproduces and improves SOTA models from top AI papers, reporting 105 surpassing variants on a curated 125-paper set.
-
LLaVA-Video: Video Instruction Tuning With Synthetic Data
LLaVA-Video-178K is a new synthetic video instruction dataset that, when combined with existing data to train LLaVA-Video, produces strong results on video understanding benchmarks.
-
From Agentic to Autogenic Network Management for AI-Native 6G and Beyond: A Standards Perspective
Autogenic network management extends agentic AI with self-programming, self-reflection, self-orienting, and self-architecting so 6G management planes can generate and evolve their own automation software at runtime.
-
EVOTS: Evolutionary Transformer Search for Time Series Forecasting
EVOTS evolves modular Transformer-like architectures for multivariate time-series forecasting and reports competitive or improved MSE versus a fixed Transformer baseline on ETT datasets across multiple settings and horizons.
-
Harvesting AI Computation at the Edge via Generic Approximation
A framework converts traditional edge tasks to NN models via NAS and schedules them on idle AI chips to improve performance without affecting primary workloads.
-
TusoAI: Agentic Optimization for Scientific Methods
TusoAI is an LLM-based agent that builds and iteratively optimizes domain-specific computational methods for scientific data analysis, outperforming expert baselines on RNA-seq denoising and earth monitoring while reporting new genetic associations.
-
On Constraint Qualifications for MPECs with Applications to Bilevel Hyperparameter Optimization for Machine Learning
Clarifies relationships among MPEC constraint qualifications and fully characterizes MPEC-LICQ for the MPEC from bilevel hyperparameter optimization in L1-loss SVM classification.
-
ORFS-agent: Tool-Using Agents for Chip Design Optimization
ORFS-agent uses LLM agents to tune parameters in chip design flows, improving geometric-mean wirelength, clock period, and co-optimization objectives by up to 2.7% over OR-AutoTuner with 40% fewer iterations on ASAP7 and SKY130HD benchmarks.
-
EPNAS: Efficient Progressive Neural Architecture Search
EPNAS uses a progressive search policy with REINFORCE performance prediction to search neural architectures in parallel, supporting multiple resource constraints and outperforming ENAS and PNAS on CIFAR-10 and ImageNet in speed and accuracy.
-
End-to-end Automated Deep Neural Network Optimization for PPG-based Blood Pressure Estimation on Wearables
An end-to-end hardware-aware optimization pipeline produces DNNs for PPG-based blood pressure estimation with up to 7.99% lower error and 83x fewer parameters that fit on ultra-low-power SoCs like GAP8.
-
Running hardware-aware neural architecture search on embedded devices under 512MB of RAM
A HW-NAS framework executable on resource-limited embedded devices generates optimized CNNs for low-end MCUs and reports state-of-the-art human-recognition accuracy on the Visual Wake Word dataset.
-
On the Difficulty of Learning a Meta-network for Training Data Selection
Analysis of MTS reveals poor GSNR and uninformative features; larger batches and distribution-based features yield 5.49% and 2.89% gains on benchmarks.
-
Exploring Vision Neural Network Pruning via Screening Methodology
A unified F-statistic screening and weighted evaluation method prunes both unstructured and structured parameters in FNNs and CNNs, claiming order-of-magnitude size reduction with competitive accuracy on vision datasets.
-
Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks
SOR-SNN employs Self-Organizing Regulation networks to reorganize a single SNN into sparse pathways, achieving better performance, energy efficiency, memory use, backward transfer, and self-repair on continual learning tasks including CIFAR100 and ImageNet.
-
Self-Adaptive 2D-3D Ensemble of Fully Convolutional Networks for Medical Image Segmentation
Self-adaptive 2D-3D FCN ensemble optimized by multiobjective evolution for prostate segmentation on PROMISE12 achieves top-10 ranking with smaller size than prior auto-designed models.
-
Deployment-Aligned Low-Precision Neural Architecture Search for Spaceborne Edge AI
Deployment-aligned low-precision NAS recovers about two-thirds of the accuracy drop from post-training quantization, achieving 0.826 mIoU on-device for a 95k-parameter model on Intel Movidius Myriad X without added complexity.
-
Bi-NAS: Towards Effective and Personalized Explanation for Recommender Systems via Bi-Level Neural Architecture Search
Bi-NAS applies bi-level NAS to search explanation architectures and LLMs for text generation, reporting gains in both recommendation accuracy and explanation effectiveness across four real-world datasets.
-
Edge-Constrained UAV Small-Object Detection with P2 Enhancement and Quantum-Inspired Lightweight Structure Search
Adding a P2 branch to YOLOX-Nano raises small-object AP by 31.10% on VisDrone; QIEA screens structures balancing accuracy, FLOPs, latency, memory and recall.
-
Genetic Deep Learning for Lung Cancer Screening
Genetic algorithm designs a CNN for lung cancer detection in CXRs achieving 97.15% accuracy, outperforming Inception-V3 and ResNet-152 with 4x and 14x fewer parameters.
-
Genetic Network Architecture Search
Genetic algorithm searches convolution cell architectures with weight sharing via SGD, reporting 96% accuracy on CIFAR10 and 80.1% on CIFAR100.
-
Spiking Neural Network Architecture Search: A Survey
A survey of Spiking Neural Network architecture search techniques viewed through a hardware/software co-design lens.