KV-cache eviction, prompt compression, recurrent state bounding, and agent memory consolidation are unified as one rate-distortion problem with a shared lower bound, shared failure mode, and transferable mechanisms.
In search of needles in a 10m haystack: Recurrent memory finds what llms miss
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
abstract
This paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark designed to assess model capabilities in extracting and processing distributed facts within extensive texts. Our evaluation, which includes benchmarks for GPT-4 and RAG, reveals that common methods are effective only for sequences up to $10^4$ elements. In contrast, fine-tuning GPT-2 with recurrent memory augmentations enables it to handle tasks involving up to $11\times 10^6$ elements. This achievement marks a substantial leap, as it is by far the longest input processed by any neural network model to date, demonstrating a significant improvement in the processing capabilities for long sequences.
citation-role summary
background 2
citation-polarity summary
roles
background 2polarities
background 2representative citing papers
TIGRAG constructs token co-occurrence graphs for scalable graph-augmented RAG and uses iterative entity-driven retrieval to improve multi-hop QA performance over dense and prior graph methods.
ARLtR is a framework for jointly constructing knowledge graphs, embeddings, and grounded QA pairs from text, released as a Roman Empire dataset with over 19,000 entities and 8,400 QA pairs.
A survey classifying RAG foundations for AIGC, summarizing enhancements, cross-modal applications, benchmarks, limitations, and future directions.
LightRAG builds graph structures into RAG indexing and retrieval with dual-level search and incremental updates to improve accuracy and speed.
citing papers explorer
-
What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
KV-cache eviction, prompt compression, recurrent state bounding, and agent memory consolidation are unified as one rate-distortion problem with a shared lower bound, shared failure mode, and transferable mechanisms.
-
Efficient Retrieval-Augmented Generation via Token Co-occurrence Graphs
TIGRAG constructs token co-occurrence graphs for scalable graph-augmented RAG and uses iterative entity-driven retrieval to improve multi-hop QA performance over dense and prior graph methods.
-
All Relations Lead to Rome: Automated Knowledge Graph Creation and Question Generation
ARLtR is a framework for jointly constructing knowledge graphs, embeddings, and grounded QA pairs from text, released as a Roman Empire dataset with over 19,000 entities and 8,400 QA pairs.
-
Retrieval-Augmented Generation for AI-Generated Content: A Survey
A survey classifying RAG foundations for AIGC, summarizing enhancements, cross-modal applications, benchmarks, limitations, and future directions.
-
LightRAG: Simple and Fast Retrieval-Augmented Generation
LightRAG builds graph structures into RAG indexing and retrieval with dual-level search and incremental updates to improve accuracy and speed.