FLUID is a continuous-time transformer using Liquid Attention Networks to model attention as stable ODE solutions that interpolate between discrete SDPA and CT-RNNs, with an explicit sink gate and liquid hyper-connections for better information flow.
Explicit sparse transformer: Concentrated attention through explicit selection
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
citation-role summary
method 1
citation-polarity summary
roles
method 1polarities
use method 1representative citing papers
VaSE improves KV cache eviction accuracy for reasoning models by over 4% versus prior eviction methods at 4x compression through value-magnitude protection and stochastic diversity.
EmambaIR is a visual state space model with cross-modal top-k sparse attention and gated SSM components that outperforms prior CNN and ViT methods on event-guided deblurring, deraining, and HDR reconstruction while reducing memory and compute costs.
Salca is a new ASIC accelerator that achieves 3.82× speedup and 74.19× energy efficiency over A100 for long-context attention via dual-compression dynamic sparse attention and pipelined hardware.
TempoNet uses a slack-quantized Transformer with deep Q-learning and sparse attention to improve deadline fulfillment rates over traditional and neural schedulers in mixed-criticality real-time workloads.
MemAgent trains an LLM with reinforcement learning to maintain a fixed-size token memory while reading a long document in chunks, and reports near-lossless QA accuracy extrapolating from 32K training context to 3.5M test tokens.
SoftMoE replaces discrete top-k routing in MoE LLMs with a truncated soft top-k LapSum relaxation, enabling differentiable optimization, learned non-uniform expert allocation across layers under a budget constraint, and comparable or better performance with fewer activations.
citing papers explorer
-
FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning
FLUID is a continuous-time transformer using Liquid Attention Networks to model attention as stable ODE solutions that interpolate between discrete SDPA and CT-RNNs, with an explicit sink gate and liquid hyper-connections for better information flow.
-
Value-Aware Stochastic KV Cache Eviction for Reasoning Models
VaSE improves KV cache eviction accuracy for reasoning models by over 4% versus prior eviction methods at 4x compression through value-magnitude protection and stochastic diversity.
-
EmambaIR: Efficient Visual State Space Model for Event-guided Image Reconstruction
EmambaIR is a visual state space model with cross-modal top-k sparse attention and gated SSM components that outperforms prior CNN and ViT methods on event-guided deblurring, deraining, and HDR reconstruction while reducing memory and compute costs.
-
Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding
Salca is a new ASIC accelerator that achieves 3.82× speedup and 74.19× energy efficiency over A100 for long-context attention via dual-compression dynamic sparse attention and pipelined hardware.
-
TempoNet: Slack-Quantized Transformer-Guided Reinforcement Scheduler for Adaptive Deadline-Centric Real-Time Dispatchs
TempoNet uses a slack-quantized Transformer with deep Q-learning and sparse attention to improve deadline fulfillment rates over traditional and neural schedulers in mixed-criticality real-time workloads.
-
MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
MemAgent trains an LLM with reinforcement learning to maintain a fixed-size token memory while reading a long document in chunks, and reports near-lossless QA accuracy extrapolating from 32K training context to 3.5M test tokens.
-
SoftMoE: Soft Differentiable Routing for Mixture-of-Experts in LLMs
SoftMoE replaces discrete top-k routing in MoE LLMs with a truncated soft top-k LapSum relaxation, enabling differentiable optimization, learned non-uniform expert allocation across layers under a budget constraint, and comparable or better performance with fewer activations.