Models multi-head transformer data flow as time-dependent Wasserstein gradient flows of an attention-capturing interaction energy, with proofs on omega-limit stationary points and stability under weight and input perturbations.
A Study on ReLU and Softmax in Transformer
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
years
2026 3verdicts
UNVERDICTED 3representative citing papers
An FPGA accelerator for a sparsity-exploiting adaptive Transformer achieves up to 2x speedup and sub-2ms latency for massive MIMO localization with under 10% accuracy loss on real measurements.
Multi-head self-attention dynamics admit a non-decreasing energy functional under suitable score-matrix conditions, with closed-form clustering thresholds and monotonic entropy production in simplified regimes.
citing papers explorer
-
Multi-Headed Transformer Architectures as Time-dependent Wasserstein Gradient Flows
Models multi-head transformer data flow as time-dependent Wasserstein gradient flows of an attention-capturing interaction energy, with proofs on omega-limit stationary points and stability under weight and input perturbations.
-
Efficient Implementation of an Adaptive Transformer Accelerator for Massive MIMO Outdoor Localization
An FPGA accelerator for a sparsity-exploiting adaptive Transformer achieves up to 2x speedup and sub-2ms latency for massive MIMO localization with under 10% accuracy loss on real measurements.
-
Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attention
Multi-head self-attention dynamics admit a non-decreasing energy functional under suitable score-matrix conditions, with closed-form clustering thresholds and monotonic entropy production in simplified regimes.