REVIEW 21 cited by
In-context Autoencoder for Context Compression in a Large Language Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
We propose the In-context Autoencoder (ICAE), leveraging the power of a large language model (LLM) to compress a long context into short compact memory slots that can be directly conditioned on by the LLM for various purposes. ICAE is first pretrained using both autoencoding and language modeling objectives on massive text data, enabling it to generate memory slots that accurately and comprehensively represent the original context. Then, it is fine-tuned on instruction data for producing desirable responses to various prompts. Experiments demonstrate that our lightweight ICAE, introducing about 1% additional parameters, effectively achieves $4\times$ context compression based on Llama, offering advantages in both improved latency and GPU memory cost during inference, and showing an interesting insight in memorization as well as potential for scalability. These promising results imply a novel perspective on the connection between working memory in cognitive science and representation learning in LLMs, revealing ICAE's significant implications in addressing the long context problem and suggesting further research in LLM context management. Our data, code and models are available at https://github.com/getao/icae.
Forward citations
Cited by 21 Pith papers
-
Out of Sight: Compression-Aware Content Protection against Agentic Crawlers
Invisible Unicode perturbations, optimized from surrogate compressors then adapted by prior-guided evolution under a low query budget, cause large information loss in agent context compression without changing human-v...
-
ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning
ReSum trains LLMs via RLVR to self-summarize reasoning trajectories, yielding 4% average performance gains and 18.6% shorter rollouts through contrastive rollout branches.
-
Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression
Independent hard prompt compression frequently retains an answer while deleting the definition or bridge needed to interpret it, and fixed-budget reinsertion of that missing support substantially improves QA accuracy.
-
Prompt Compression via Activation Aggregation
A learned weighted sum of intermediate-layer activations compresses an instruction prompt into a single patch vector that, injected at an early layer, recovers task accuracy within ~2% of the full prompt.
-
What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
KV-cache eviction, prompt compression, recurrent state bounding, and agent memory consolidation are unified as one rate-distortion problem with a shared lower bound, shared failure mode, and transferable mechanisms.
-
Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Sleep-time Knowledge Seeding plus Dreaming lets LLMs expand capacity, distill fragile in-context memories into stable parameters, and self-improve without human labels.
-
When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agents
The paper claims prompt compression is a new attack surface, but the abstract's COMA attack never appears in the body and the body's SoftCom requires white-box access.
-
ARC-Encoder: learning compressed text representations for large language models
ARC-Encoder pools queries in an encoder's last attention layer to produce compressed continuous representations that a frozen decoder consumes as token embeddings.
-
MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Prompts
MHA-RAG encodes retrieved exemplars into order-invariant soft prompts via multi-head attention, claiming ~20-point effective-accuracy gains over RAG at ~10x lower inference FLOPs.
-
Cartridges: Lightweight and general-purpose long context representations via self-study
A per-corpus trained KV cache, called a Cartridge, matches full-context in-context learning quality on long-document benchmarks while using up to 38.6x less serving memory.
-
GEM: Empowering LLM for both Embedding Generation and Language Understanding
GEM fine-tunes decoder-only LLMs with bottleneck special tokens and a mixed next-token prediction and contrastive objective, giving them text-embedding ability with only a modest MMLU drop.
-
Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention
HyCo2 combines soft global compression with hard local token selection, reporting QA performance near uncompressed retrieval while cutting context tokens by about 88.8%.
-
Reranking with Compressed Document Representation
A reranker trained on 8-token PISCO document embeddings plus a short query achieves near-identical nDCG@10 to full-text rerankers on BeIR and TREC-DL while running up to 16x faster.
-
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs
A frozen LLM can process very long contexts by recurrently compressing them with a Perceiver and injecting the compressed memory through gated cross-attention, with query-dependent compression boosting QA performance.
-
C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness
C3oT uses prompt-conditioned fine-tuning on both long and short chain-of-thought data to generate about 50 percent shorter reasoning traces with roughly unchanged accuracy.
-
ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution
ZUNA1.1, an open-source 380M EEG diffusion autoencoder, reconstructs variable-length, flexibly masked EEG at least as well as its predecessor and far better than spherical spline interpolation.
-
Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoning
ASCoT claims later reasoning errors are more harmful than early ones and uses a position-weighted verifier to prune and correct CoT steps, but its key evidence is internally inconsistent.
-
Lossless Token Sequence Compression via Meta-Tokens
A new compression scheme replaces repeated token subsequences with learnable placeholder tokens, shrinking prompts by 15-27% with no loss of information, and fine-tuned LLMs perform nearly as well as on uncompressed input.
-
The Scaling Law for LoRA Base on Mutual Information Upper Bound
The claimed mutual information upper bound for LoRA scaling laws is invalid because the key inequality in the proof is false.
-
Token-Operations-Oriented Inference Optimization Techniques for Large Models
The paper introduces a four-layer technical architecture for token-operations-oriented inference optimization in large models and reviews key technologies and industry status at each layer.
-
An Enhanced Text Compression Approach Using Transformer-based Language Models
Removing vowels before LZW compression yields high compression ratios, but the resulting text cannot be restored without a large transformer, making the claimed state-of-the-art comparison unfair.
Discussion (0). Continue with ORCID to comment.