REVIEW 6 cited by
Augmenting Language Models with Long-Term Memory
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Existing large language models (LLMs) can only afford fix-sized inputs due to the input length limit, preventing them from utilizing rich long-context information from past inputs. To address this, we propose a framework, Language Models Augmented with Long-Term Memory (LongMem), which enables LLMs to memorize long history. We design a novel decoupled network architecture with the original backbone LLM frozen as a memory encoder and an adaptive residual side-network as a memory retriever and reader. Such a decoupled memory design can easily cache and update long-term past contexts for memory retrieval without suffering from memory staleness. Enhanced with memory-augmented adaptation training, LongMem can thus memorize long past context and use long-term memory for language modeling. The proposed memory retrieval module can handle unlimited-length context in its memory bank to benefit various downstream tasks. Typically, LongMem can enlarge the long-form memory to 65k tokens and thus cache many-shot extra demonstration examples as long-form memory for in-context learning. Experiments show that our method outperforms strong long-context models on ChapterBreak, a challenging long-context modeling benchmark, and achieves remarkable improvements on memory-augmented in-context learning over LLMs. The results demonstrate that the proposed method is effective in helping language models to memorize and utilize long-form contents. Our code is open-sourced at https://aka.ms/LongMem.
Forward citations
Cited by 6 Pith papers
-
A-TMA: Decoupling State-Aware Memory Failures in Long-Term Agent Memory
Explicit bank, retrieval, and QA state roles reduce ghost-memory failures on conflict-heavy LTP and improve some temporal scores on LoCoMo, with host-dependent gains.
-
Memory-Managed Long-Context Attention: Bounded Editable Memory with a Hard Lifecycle and Calibrated Sparse Fallback
Editable bounded memory slots with lifecycle control plus sparse fallback cover overwrite and no-signal long-context cases that pure fixed-state or pure sparse methods fail under controlled conditions.
-
LANTERN: Layered Archival and Temporal Episodic Retrieval Network for Long-Context LLM Conversations
Extractive archival plus hybrid retrieval recovers 78% of facts lost to conversation compaction, beating MemGPT-style LLM extraction at near-zero inference cost.
-
The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMs
Adding user memory to LLMs degrades their emotional-intelligence test scores and systematically disadvantages marginalized user profiles.
-
Towards General Continuous Memory for Vision-Language Models
A vision-language model can act as its own continuous memory encoder, compressing external multimodal knowledge into eight embeddings that improve reasoning when prepended to the frozen model.
-
Memory for Large Language Models
The paper organizes LLM memory architectures along three axes—implicit vs explicit representation, offline vs online update, and short vs long-term persistence—plus finer-grained update rules.
Discussion (0). Sign in to comment.