REVIEW 18 cited by
PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answering
read the original abstract
Long-term memory plays a critical role in personal interaction, considering long-term memory can better leverage world knowledge, historical information, and preferences in dialogues. Our research introduces PerLTQA, an innovative QA dataset that combines semantic and episodic memories, including world knowledge, profiles, social relationships, events, and dialogues. This dataset is collected to investigate the use of personalized memories, focusing on social interactions and events in the QA task. PerLTQA features two types of memory and a comprehensive benchmark of 8,593 questions for 30 characters, facilitating the exploration and application of personalized memories in Large Language Models (LLMs). Based on PerLTQA, we propose a novel framework for memory integration and generation, consisting of three main components: Memory Classification, Memory Retrieval, and Memory Synthesis. We evaluate this framework using five LLMs and three retrievers. Experimental results demonstrate that BERT-based classification models significantly outperform LLMs such as ChatGLM3 and ChatGPT in the memory classification task. Furthermore, our study highlights the importance of effective memory integration in the QA task.
Forward citations
Cited by 18 Pith papers
-
Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data
Setoka evaluates memory-augmented agents on four levels of user understanding—semantic memory, episodic memory, behavior patterns, and personality traits—over synthesized heterogeneous user data, and finds performance...
-
Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents
DCPM reorganizes LLM agent memory into a cognitive hierarchy driven by a synchronous daytime belief writer and an asynchronous nighttime schema engine, reporting gains on cross-session inference benchmarks.
-
MRMMIA: Membership Inference Attacks on Memory in Chat Agents
MRMMIA is a multi-recall-probe membership inference attack that extracts signals from chat agent memory and outperforms baselines in black-, gray-, and white-box settings.
-
MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models
MemLens benchmark shows long-context LVLMs lose accuracy with length while memory agents lose visual fidelity, with multi-session reasoning below 30% for most systems and neither approach solving the task alone.
-
LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
LongMemEval-V2 is a new benchmark where AgentRunbook-C reaches 72.5% accuracy on long-term agent memory tasks, beating RAG baselines at 48.5% and basic coding agents at 69.3%.
-
PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments
PERMA is a new benchmark using temporally ordered events, text variability, and linguistic alignment to evaluate LLM memory agents on persona consistency beyond simple retrieval.
-
LMEB: Long-horizon Memory Embedding Benchmark
LMEB benchmark shows that embedding models' performance on traditional retrieval does not transfer to long-horizon memory tasks, larger models do not always perform better, and LMEB measures capabilities orthogonal to MTEB.
-
Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data
Existing agent memory systems score well on single-fact recall but fall near chance on behavior-pattern and personality-trait inference over heterogeneous synthetic user data.
-
LATTE: Forecasting Peer Anchored Preference Trajectories for Personalized LLM Generation
LATTE improves personalized LLM generation by forecasting peer-anchored relative preference trajectories and injecting the forecast via a State to Token Bridge, raising ROUGE-L from 0.219-0.245 to 0.259 on Amazon Revi...
-
Anticipate and Learn: Unleashing Idle-Time Compute in Proactive Agents
ProAct uses idle compute to anticipate user needs via dialogue history and memory, achieving 14.8% fewer turns, 11.7% less user effort, and 28.1% fewer hallucinations than reactive baselines on the new ProActEval benchmark.
-
HingeMem: Boundary Guided Long-Term Memory with Query Adaptive Retrieval for Scalable Dialogues
HingeMem segments dialogue memory via boundary-triggered hyperedges over four elements and applies query-adaptive retrieval, yielding ~20% relative gains and 68% lower QA token cost versus baselines on LOCOMO.
-
SelRoute: Query-Type-Aware Routing for Long-Term Conversational Memory Retrieval
SelRoute routes queries to type-specific retrieval pipelines, achieving Recall@5 of 0.800 with a 109M model on LongMemEval_M and outperforming LLM-augmented baselines including a strong zero-ML lexical method.
-
LMEB: Long-horizon Memory Embedding Benchmark
LMEB is a new benchmark of 193 retrieval tasks spanning episodic, dialogue, semantic, and procedural memory on which top embedding models score about 61 NDCG@10, largely uncorrelated with MTEB.
-
LMEB: Long-horizon Memory Embedding Benchmark
LMEB is a new benchmark that evaluates embedding models on long-horizon memory retrieval and shows this skill is largely orthogonal to traditional passage-retrieval performance.
-
LMEB: Long-horizon Memory Embedding Benchmark
LMEB is a 22-dataset, 193-task zero-shot benchmark showing that long-horizon memory retrieval is hard, not solved by scale, and largely orthogonal to MTEB passage-retrieval skill.
-
MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning
MemSifter trains a 4B proxy with an outcome-driven, rank-sensitive RL reward to sift LLM memory, and on eight benchmarks it matches or beats embedding, graph, and long-context baselines.
-
EngramaBench: Evaluating Long-Term Conversational Memory with Structured Graph Retrieval
EngramaBench shows structured graph memory outperforms full-context prompting on cross-space reasoning in long conversations but scores lower overall than full-context and higher than vector retrieval.
-
A Survey of Context Engineering for Large Language Models
The survey organizes Context Engineering into retrieval, processing, management, and integrated systems like RAG and multi-agent setups while identifying an asymmetry where LLMs handle complex inputs well but struggle...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.