Proves GD convergence to stationary point neighborhoods for general NN architectures beyond NTK via block-level analysis, analyticity, and local smoothness conditions.
L2 Regularization for Learning Kernels
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
abstract
The choice of the kernel is critical to the success of many learning algorithms but it is typically left to the user. Instead, the training data can be used to learn the kernel by selecting it out of a given family, such as that of non-negative linear combinations of p base kernels, constrained by a trace or L1 regularization. This paper studies the problem of learning kernels with the same family of kernels but with an L2 regularization instead, and for regression problems. We analyze the problem of learning kernels with ridge regression. We derive the form of the solution of the optimization problem and give an efficient iterative algorithm for computing that solution. We present a novel theoretical analysis of the problem based on stability and give learning bounds for orthogonal kernels that contain only an additive term O(pp/m) when compared to the standard kernel ridge regression stability bound. We also report the results of experiments indicating that L1 regularization can lead to modest improvements for a small number of kernels, but to performance degradations in larger-scale cases. In contrast, L2 regularization never degrades performance and in fact achieves significant improvements with a large number of kernels.
verdicts
UNVERDICTED 5representative citing papers
A spectrum-aware decision boundary algorithm enables effective hard-label black-box adversarial attacks on 3D point cloud models by fusing spectral information across classes and performing curvature-aware iterative optimization.
ASTRA disentangles subject identity from pose structure in diffusion transformers via retrieval-augmented pose guidance, asymmetric EURoPE embeddings, and a DSM adapter to improve multi-subject generation.
An iterative procedure computes optimal ridge regularization from generative parameters in fixed-X settings and achieves near-optimal generalization with sample-based estimates across sample sizes, aspect ratios, and noise levels.
Variational autoencoders generate higher-resolution dual-channel visual memory schema maps that separately predict true and false memorability, extending prior CNN approaches.
citing papers explorer
-
Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime
Proves GD convergence to stationary point neighborhoods for general NN architectures beyond NTK via block-level analysis, analyticity, and local smoothness conditions.
-
Hard-Label Black-Box Attacks on 3D Point Clouds
A spectrum-aware decision boundary algorithm enables effective hard-label black-box adversarial attacks on 3D point cloud models by fusing spectral information across classes and performing curvature-aware iterative optimization.
-
ASTRA: Enhancing Multi-Subject Generation with Retrieval-Augmented Pose Guidance and Disentangled Position Embedding
ASTRA disentangles subject identity from pose structure in diffusion transformers via retrieval-augmented pose guidance, asymmetric EURoPE embeddings, and a DSM adapter to improve multi-subject generation.
-
Optimal ridge regularization revisited
An iterative procedure computes optimal ridge regularization from generative parameters in fixed-X settings and achieves near-optimal generalization with sample-based estimates across sample sizes, aspect ratios, and noise levels.
-
Predicting Visual Memory Schemas with Variational Autoencoders
Variational autoencoders generate higher-resolution dual-channel visual memory schema maps that separately predict true and false memorability, extending prior CNN approaches.