DARE reuses up to 87% of attention activations in diffusion LLMs through KV caching and output reuse, delivering 1.2x per-layer latency gains with average performance drops of 1.2-2.0%.
Training transformers with enforced lipschitz constants
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
years
2026 7roles
background 1polarities
background 1representative citing papers
Discrete tokenization in scientific foundation models imposes a geometric alignment tax that distorts continuous manifolds, with continuous heads reducing distortion by up to 8.5x and exposing three failure regimes in 14 biological models.
Deeper transformer layers produce intermediate representations that are harder to lossy-compress, and the paper links this to growing covariance and Rademacher complexity.
Splitting weight matrices into a fixed-norm direction and learnable per-row/column magnitudes improves LLM training over AdamW/Muon, removes weight decay and warmup, and transfers the optimal LR across width.
Entry-wise clipping achieves spectral control of gradients via localization under heavy-tailed contamination, with O(ε^{-4}) convergence and empirical savings on NanoGPT pretraining.
Muon optimizer in adversarial training imposes spectral-norm stability on matrix updates and matches or exceeds SGD/AdamW robustness on CNNs and ViTs under lp attacks.
Proximal stochastic spectral preconditioning converges for nonconvex constrained objectives under heavy-tailed noise, with a variance-reduced version achieving faster rates and a refined analysis of Muon iterations.
citing papers explorer
-
DARE: Diffusion Language Model Activation Reuse for Efficient Inference
DARE reuses up to 87% of attention activations in diffusion LLMs through KV caching and output reuse, delivering 1.2x per-layer latency gains with average performance drops of 1.2-2.0%.
-
The Geometric Alignment Tax: Tokenization vs. Continuous Geometry in Scientific Foundation Models
Discrete tokenization in scientific foundation models imposes a geometric alignment tax that distorts continuous manifolds, with continuous heads reducing distortion by up to 8.5x and exposing three failure regimes in 14 biological models.
-
Understanding Rate-Distortion Performance in Distributed Transformer Inference
Deeper transformer layers produce intermediate representations that are harder to lossy-compress, and the paper links this to growing covariance and Rademacher complexity.
-
Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors
Splitting weight matrices into a fixed-norm direction and learnable per-row/column magnitudes improves LLM training over AdamW/Muon, removes weight decay and warmup, and transfers the optimal LR across width.
-
Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?
Entry-wise clipping achieves spectral control of gradients via localization under heavy-tailed contamination, with O(ε^{-4}) convergence and empirical savings on NanoGPT pretraining.
-
When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
Muon optimizer in adversarial training imposes spectral-norm stability on matrix updates and matches or exceeds SGD/AdamW robustness on CNNs and ViTs under lp attacks.
-
Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
Proximal stochastic spectral preconditioning converges for nonconvex constrained objectives under heavy-tailed noise, with a variance-reduced version achieving faster rates and a refined analysis of Muon iterations.