REVIEW 2 cited by
Semantic Caching of Contextual Summaries for Efficient Question-Answering with Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Semantic Caching of Contextual Summaries for Efficient Question-Answering with Language Models
read the original abstract
Large Language Models (LLMs) are increasingly deployed across edge and cloud platforms for real-time question-answering and retrieval-augmented generation. However, processing lengthy contexts in distributed systems incurs high computational overhead, memory usage, and network bandwidth. This paper introduces a novel semantic caching approach for storing and reusing intermediate contextual summaries, enabling efficient information reuse across similar queries in LLM-based QA workflows. Our method reduces redundant computations by up to 50-60% while maintaining answer accuracy comparable to full document processing, as demonstrated on NaturalQuestions, TriviaQA, and a synthetic ArXiv dataset. This approach balances computational cost and response quality, critical for real-time AI assistants.
Forward citations
Cited by 2 Pith papers
-
Mobility-Aware Cache Framework for Scalable LLM-Based Human Mobility Simulation
A latent-space reasoning cache with a lightweight decoder cuts the cost of LLM-based human mobility simulation by roughly 40-90% while keeping trajectory quality comparable.
-
From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching
An attacker can force false cache-key collisions in LLM semantic caching and hijack responses or agent tool calls, with 86% hit rate in experiments.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.