REVIEW 10 cited by
Differential Privacy Mechanisms in Neural Tangent Kernel Regression
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Training data privacy is a fundamental problem in modern Artificial Intelligence (AI) applications, such as face recognition, recommendation systems, language generation, and many others, as it may contain sensitive user information related to legal issues. To fundamentally understand how privacy mechanisms work in AI applications, we study differential privacy (DP) in the Neural Tangent Kernel (NTK) regression setting, where DP is one of the most powerful tools for measuring privacy under statistical learning, and NTK is one of the most popular analysis frameworks for studying the learning mechanisms of deep neural networks. In our work, we can show provable guarantees for both differential privacy and test accuracy of our NTK regression. Furthermore, we conduct experiments on the basic image classification dataset CIFAR10 to demonstrate that NTK regression can preserve good accuracy under a modest privacy budget, supporting the validity of our analysis. To our knowledge, this is the first work to provide a DP guarantee for NTK regression.
Forward citations
Cited by 10 Pith papers
-
Curse of Attention: A Kernel-Based Perspective for Why Transformers Fail to Generalize on Time Series Forecasting and Beyond
A theoretical analysis claims attention networks fail to learn residual features when time series steps have opposite signs, giving a possible explanation for the known advantage of linear residual models.
-
How Private is Your Attention? Bridging Privacy with In-Context Learning
Differentially private pretraining of linear attention heads for in-context linear regression has excess risk that decays like 1/(N L^3) in low dimensions and D^2/(N^2 L^2) in high dimensions, up to log factors and pr...
-
Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform
A new algorithm computes RoPE attention in n^{1+o(1)} time under bounded entries by decomposing the attention matrix into a small sum of rescaled Toeplitz matrices, then using FFT.
-
LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers
LazyDiT learns small gates that decide when to reuse cached layer outputs, cutting diffusion transformer compute by up to half while matching or beating DDIM quality.
-
Numerical Pruning for Efficient Autoregressive Models
A Newton's method derived importance score plus a constrained least-squares compensation prunes LLaMA and LlamaGen models with reported gains over LLM-Pruner, SliceGPT, and FLAP.
-
Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapse
A residual self-attention network with all weight entries bounded by a small η can be approximated by one layer to error O(η)‖X‖∞, so skip connections do not prevent layer collapse.
-
Force Matching with Relativistic Constraints: A Physics-Inspired Approach to Stable and Efficient Generative Modeling
Force Matching replaces velocity matching in flow-based generative models with a relativistic force objective, but the toy experiments are designed so the model class matches the data generator exactly.
-
Universal Approximation of Visual Autoregressive Transformers
The paper's headline claim that VAR transformers universally approximate all Lipschitz image maps is not supported, because the theorem restricts the target class and its key lemma has an invalid linearity step.
-
High-Order Matching for One-Step Shortcut Diffusion Models
HOMO extends shortcut diffusion with acceleration and jerk supervision, but the proof of superior approximation is not supported and experiments lack error bars.
-
On Computational Limits and Provably Efficient Criteria of Visual Autoregressive Models: A Fine-Grained Complexity Analysis
Under SETH, the paper claims VAR models cannot be approximated faster than O(n^4) when attention entries are Theta(sqrt(log n)), but can be approximated in O(n^{2+o(1)}) when entries are o(sqrt(log n)).
Discussion (0). Continue with ORCID to comment.