REVIEW 15 cited by
Efficient Attentions for Long Document Summarization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The quadratic computational and memory complexities of large Transformers have limited their scalability for long document summarization. In this paper, we propose Hepos, a novel efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source. We further conduct a systematic study of existing efficient self-attentions. Combined with Hepos, we are able to process ten times more tokens than existing models that use full attentions. For evaluation, we present a new dataset, GovReport, with significantly longer documents and summaries. Results show that our models produce significantly higher ROUGE scores than competitive comparisons, including new state-of-the-art results on PubMed. Human evaluation also shows that our models generate more informative summaries with fewer unfaithful errors.
Forward citations
Cited by 15 Pith papers
-
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Under dynamic W4A4KV4 quantisation, a head-shared RoPE-aligned pairwise rotation that exactly minimizes a pooled variance surrogate still yields higher perplexity than full-head Hadamard mixing in all evaluated comparisons.
-
CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation
Offline-learned head-reliability and risk-threshold tables make prefill-only KV compression recover about 97.7% of uncompressed LongBench accuracy at a 512-token-per-layer memory budget.
-
EvolKV: Evolutionary KV Cache Compression for LLM Inference
CMA-ES search over per-layer KV cache budgets beats uniform and pyramidal compression heuristics on LongBench, NIAH, RULER, and GSM8K, and edges past the full cache on one code dataset at 1.5% of the budget.
-
Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification
AASD combines alignment sampling from the prefilling distribution with entropy-based conditional verification to improve both the accuracy and speed of retrieval-based speculative decoding.
-
CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective
A new selection metric that combines attention weights with projected value-state norms reduces output perturbation in LLM key-value cache eviction.
-
Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads
Task-KV identifies 'heterogeneous' attention heads by distance from a per-task semantic center and allocates differentiated KV cache budgets, achieving modest average gains on long-context benchmarks at reduced memory.
-
AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference
AdaSkip speeds up long-context LLM inference by adaptively skipping low-importance attention and FFN sublayers in both the prompt-reading and token-generation phases, with quality tradeoffs.
-
CaseSumm: A Large-Scale Dataset for Long-Context Summarization from U.S. Supreme Court Opinions
CaseSumm, a 25.6K-pair dataset of Supreme Court opinions and official syllabuses, shows automated metrics favor fine-tuned Mistral while human experts prefer GPT-4, and LLM judges do not align with humans better than ROUGE.
-
RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory
RED-PIM restructures attention to use a compact d×d intermediate matrix, reducing PIM inter-bank data movement from O(N²) to O(N) and claiming up to 99.99% latency reduction in simulation.
-
Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization
FoPE replaces RoPE's single-frequency rotation per dimension with a Fourier series and clips under-trained low frequencies, improving length generalization in transformer LMs.
-
Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models
Giving each attention head its own RoPE base frequency lets a single continual-pretraining stage at 128k match or beat a three-stage schedule, per the paper's NiaH, PPL and RULER results.
-
From Global to Local: Learning Context-Aware Graph Representations for Document Classification and Summarization
Replacing full attention with sliding-window attention when building document sentence-graphs gives competitive classification accuracy at lower attention cost, with unfiltered local graphs best on medium-length news.
-
HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
HoPE replaces RoPE's sine/cosine rotations with hyperbolic functions plus an exponential damping term to enforce monotonic attention decay, but the claimed consistent superiority and the 'RoPE as special case' theorem...
-
PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference
PagedEviction prunes the KV cache in whole blocks using a key-value norm ratio, speeding up long-context LLM inference in vLLM while staying close to full-cache accuracy.
-
More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
Storing 4x as many KV cache tokens at 4-bit precision outperforms storing 1x tokens at 16-bit precision on long-context benchmarks at equal data-memory budgets.
Discussion (0). Continue with ORCID to comment.