Pith. sign in

REVIEW 18 cited by

Efficient Attentions for Long Document Summarization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2104.02112 v2 pith:O5RQ5RNL submitted 2021-04-05 cs.CL

Luyang Huang , Shuyang Cao , Nikolaus Parulian , Heng Ji , Lu Wang This is my paper
classification cs.CL
keywords efficientmodelsattentionsdocumentevaluationexistingheposlong
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The quadratic computational and memory complexities of large Transformers have limited their scalability for long document summarization. In this paper, we propose Hepos, a novel efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source. We further conduct a systematic study of existing efficient self-attentions. Combined with Hepos, we are able to process ten times more tokens than existing models that use full attentions. For evaluation, we present a new dataset, GovReport, with significantly longer documents and summaries. Results show that our models produce significantly higher ROUGE scores than competitive comparisons, including new state-of-the-art results on PubMed. Human evaluation also shows that our models generate more informative summaries with fewer unfaithful errors.

Discussion (0). Sign in to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Larch: Learned Query Optimization for Semantic Predicates

    cs.DB 2026-06 unverdicted novelty 6.0 of 10

    Larch uses a GNN-MDP formulation and a selectivity predictor plus dynamic programming to reorder semantic filter evaluation, cutting token usage 3x-19x versus prior systems on real and synthetic workloads.

  2. Reformulating KV Cache Eviction Problem for Long-Context LLM Inference

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    LaProx reformulates KV cache eviction as an output-aware matrix approximation, enabling a unified global token selection strategy that preserves LLM performance at 5% cache size across long-context benchmarks.

  3. FlashRT: Towards Computationally and Memory Efficient Red-Teaming for Prompt Injection and Knowledge Corruption

    cs.CR 2026-04 unverdicted novelty 6.0 of 10

    FlashRT delivers 2x-7x speedup and 2x-4x GPU memory reduction for prompt injection and knowledge corruption attacks on long-context LLMs versus nanoGCG.

  4. SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference

    cs.NI 2026-04 unverdicted novelty 6.0 of 10

    SparKV reduces time-to-first-token by 1.3x-5.1x and energy use by 1.5x-3.3x for on-device LLM inference by adaptively choosing between cloud KV streaming and local computation while overlapping execution and adjusting...

  5. CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    CompilerKV uses offline-compiled retention tables as portable priors to achieve SOTA prefill-only KV compression performance across backbones at low token budgets.

  6. CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

    cs.LG 2026-02 conditional novelty 6.0 of 10

    Offline-learned head-reliability and risk-threshold tables make prefill-only KV compression recover about 97.7% of uncompressed LongBench accuracy at a 512-token-per-layer memory budget.

  7. EvolKV: Evolutionary KV Cache Compression for LLM Inference

    cs.LG 2025-09 conditional novelty 6.0 of 10

    CMA-ES search over per-layer KV cache budgets beats uniform and pyramidal compression heuristics on LongBench, NIAH, RULER, and GSM8K, and edges past the full cache on one code dataset at 1.5% of the budget.

  8. Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference

    cs.CL 2024-07 accept novelty 6.0 of 10

    Ada-KV is the first head-wise adaptive KV cache budget allocator for LLMs, using a theoretical loss upper bound to allocate eviction differently per attention head and yielding higher quality than uniform methods on l...

  9. Retentive Network: A Successor to Transformer for Large Language Models

    cs.CL 2023-07 unverdicted novelty 6.0 of 10

    RetNet is a new sequence modeling architecture that delivers parallel training, constant-time inference, and competitive language modeling performance as a potential replacement for Transformers.

  10. RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory

    cs.LG 2026-07 conditional novelty 5.0 of 10

    RED-PIM restructures attention to use a compact d×d intermediate matrix, reducing PIM inter-bank data movement from O(N²) to O(N) and claiming up to 99.99% latency reduction in simulation.

  11. Demystifying the Design Space and Best Practices for Heterogeneous LLM Inference and Serving

    cs.DC 2026-06 unverdicted novelty 5.0 of 10

    Organizes the heterogeneous LLM prefill-decode design space along four axes and extracts three boundary decisions with guidance on precision, KV representation, and ownership.

  12. Demystifying the Design Space and Best Practices for Heterogeneous LLM Inference and Serving

    cs.DC 2026-06 unverdicted novelty 5.0 of 10

    The paper organizes heterogeneous prefill-decode LLM serving into a four-axis design space and identifies three recurring boundary decisions that require joint choices.

  13. Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

    cs.CL 2026-06 unverdicted novelty 5.0 of 10

    K-VEC is a coverage-aware KV-cache eviction strategy using cross-head and cross-layer modules that improves performance by up to 10.35 points over prior methods on LongBench subsets at fixed memory budget.

  14. How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignment

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    Shadow Mask Distillation enables KV cache compression in RL post-training of LLMs by mitigating amplified off-policy bias that defeats standard importance reweighting.

  15. DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    DepthKV allocates a fixed global KV cache budget across LLM layers based on per-layer pruning sensitivity, outperforming uniform pruning at the same overall budget.

  16. From Global to Local: Learning Context-Aware Graph Representations for Document Classification and Summarization

    cs.CL 2026-02 conditional novelty 4.0 of 10

    Replacing full attention with sliding-window attention when building document sentence-graphs gives competitive classification accuracy at lower attention cost, with unfiltered local graphs best on medium-length news.

  17. HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models

    cs.CL 2025-09 reject novelty 4.0 of 10

    HoPE replaces RoPE's sine/cosine rotations with hyperbolic functions plus an exponential damping term to enforce monotonic attention decay, but the claimed consistent superiority and the 'RoPE as special case' theorem...

  18. PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference

    cs.LG 2025-09 conditional novelty 4.0 of 10

    PagedEviction prunes the KV cache in whole blocks using a key-value norm ratio, speeding up long-context LLM inference in vLLM while staying close to full-cache accuracy.

Pith tools