KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing

Dongqi Fu; Haoyu Wang; Hong Li; Hong Yan; Mufei Li; Pan Li; Shikun Liu; Yinglong Xia

A learned local edit to an LLM’s key–value cache can make the model behave as if a deleted span never appeared, nearly matching full recomputation at far lower cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 17:47 UTC pith:WO5U2Y7T

load-bearing objection Solid systems paper: learned local KV steering nearly matches full recompute for single-span post-hoc erasing with much better latency scaling. the 3 major comments →

arxiv 2606.17034 v2 pith:WO5U2Y7T submitted 2026-06-15 cs.CL cs.LG

KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing

Mufei Li , Shikun Liu , Dongqi Fu , Haoyu Wang , Yinglong Xia , Hong Li , Hong Yan , Pan Li This is my paper

classification cs.CL cs.LG

keywords KV cachecontext erasinglong-context LLMscache editingsteering statesretrieval-augmented generationpost-prefill correction

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Once a long context has been written into an LLM’s key–value cache, deleting a short bad span is expensive: that span has already contaminated every later token’s cached states, so exact erasure must recompute the whole suffix. This paper argues that you do not need that full recompute. Instead, a small trainable “eraser” can replace only the erased span’s cache entries with learned steering states and leave the rest of the cache untouched, so future decoding still matches what the model would have done on the cleaned prompt. Trained first on generic span-neighbor tasks and then fine-tuned for erasing needles and factual distractors, the method nearly matches full recomputation from 1K to 32K tokens while its latency grows by only about a quarter rather than nearly eighteenfold. On unseen long-document QA with harmful distractors it is the strongest approximate method and runs three to four times faster than exact recompute. The practical stake is post-prefill correction of stale retrievals, wrong tool results, retracted preferences, and prompt injections without paying suffix-length cost.

Core claim

KVEraser shows that post-hoc context erasing can be done in KV space by replacing only the erased interval with learned steering key–value states and reusing the contaminated suffix cache unchanged. The frozen generator, decoding from this surrogate cache, nearly recovers the behavior of full recomputation on the edited prompt across long contexts, and the approach transfers to unseen QA settings with harmful factual distractors while keeping cost proportional to the erased span rather than the suffix.

What carries the argument

Surrogate cache construction: a trainable eraser (a copy of the generator backbone without the LM head), conditioned on the preserved prefix cache and the erased span, emits length-preserving steering KV states that overwrite only positions m…n; the prefix and suffix caches are reused as-is so that decoding approximates the counterfactual prompt with the span deleted.

Load-bearing premise

A short, length-preserving steering block built only from the kept prefix and the span to erase can cancel contamination already baked into an arbitrarily long reused suffix.

What would settle it

On held-out long-context erasing tasks (or multi-span and harder out-of-distribution edits), measure whether exact-match or answer quality collapses toward delete-and-shift or local suffix-repair baselines as average suffix length grows while the erased span stays short; a large gap to full recompute would falsify the claim that local steering is enough.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid systems paper: learned local KV steering nearly matches full recompute for single-span post-hoc erasing with much better latency scaling.

read the letter

The useful takeaway is simple: they treat post-hoc span deletion as a local learned edit on the KV cache rather than suffix recompute or instruction-only forgetting, and on the reported single-span setup it works. Replace only the erased interval with steering states from a frozen-generator-compatible eraser, keep the rest of the cache, and train so decoding matches the clean edited prompt.

What is new is the problem framing plus the method. Shared-prefix reuse and eviction do not apply when the span is invalid but influential and has already contaminated the suffix. They give a clean formalization (surrogate cache, functional success criterion), a length-preserving local edit, and a two-stage curriculum (span-neighbor pretrain on Wikipedia inserts, then NIAH + distractor QA fine-tune). Empirically this is the strongest part: on held-out NIAH from 1K–32K they match full recompute exact match while latency grows ~24% vs 17.6× for recompute; on three unseen QA sets they sit on the approximate Pareto frontier at 3–4× speedup. Baselines (delete-and-shift, instruction-only, two 15% suffix repairs) and conditioning ablations are the right comparisons, and failure breakdowns are honest.

Soft spots are real but scoped. The load-bearing premise is that a prefix+span-conditioned steering block can cancel contamination already baked into an arbitrary reused suffix. All positive evidence is single contiguous ~100-token Wikipedia-style or synthetic needles; multi-span is explicitly out of scope, residual gap to full recompute remains on natural QA, and there is one backbone (Qwen3-8B) with modest fine-tune data and no public code/data. The stress-test concern about multi-hop residual contamination is fair as a generalization limit, not as a hole in the reported claims—the paper already shows partial repair fails and does not overclaim multi-span. Math is complexity intuition plus teacher-forced NLL, not a proof of exact cancellation; citations to reuse/eviction work are appropriate.

This is for people building long-context serving, RAG, and agents who care about stale retrieval, bad tool results, or retracted preferences after prefill. Worth a serious referee. I would engage and cite for the systems idea and the scaling numbers; I would not treat the local-steering premise as settled beyond single-span Wikipedia-like edits.

Referee Report

3 major / 0 minor

Summary. The paper studies post-hoc context erasing after KV prefill: a short span e is identified as stale, wrong, or harmful only after the full context has been cached, so suffix states are already contaminated. Exact erasing reuses the valid prefix and recomputes the entire suffix under the edited prompt, with cost governed by suffix length. KVEraser instead builds a surrogate cache by replacing only the erased interval with learned, length-preserving steering KV states (conditioned on the preserved prefix cache and e) while reusing the remaining cache unchanged. An eraser module, initialized as a trainable copy of the generator backbone, is trained with a two-stage pipeline (generic span-neighbor pre-training, then task-specific fine-tuning) so that decoding from the surrogate cache matches the frozen model on the clean edited prompt. On a controlled multi-value NIAH erasing benchmark (1K–32K), KVEraser nearly matches full recompute in exact match while latency grows only ~24% versus ~17.6× for full recompute; on three unseen long-document QA sets with harmful factual distractors it is the best approximate method at 3–4× lower latency than full recompute.

Significance. Post-hoc localized context erasing is a practically important and under-addressed operation for long-context RAG, tool use, preference retraction, and injection defense. The paper cleanly separates functional behavioral match (Eq. 2) from tokenwise cache reconstruction, and shows that a local length-preserving steering block can be competitive with full suffix recomputation under a well-scoped single-span setting. Strengths include a clear complexity argument (Sec. 4.2), appropriate approximate baselines (delete-and-shift, instruction-only forgetting, two local suffix-repair variants), scaling plots across six context sizes, transfer to three held-out QA datasets, conditioning ablations (Table 1), and failure-mode breakdowns. If the result holds under broader span lengths and serving constraints, learned local KV steering is a useful addition to the KV-cache management toolkit alongside reuse and eviction methods.

major comments (3)

Sec. 4.1 and Sec. 5.1–5.2: training and evaluation almost exclusively use short erased spans (~100-token Wikipedia chunks or short synthetic needles). The central efficiency claim (latency +24% from 1K to 32K; O(|e|(|p|+|e|)) vs O(|s|(|p|+|s|)) in Sec. 4.2) is stated for the regime |e| ≪ |s|. Without a systematic sweep of erased-span length and erase position (early vs late), it remains unclear how far the reported quality–efficiency frontier extends when the invalidated span is longer (e.g., multi-paragraph tool output) or when the average suffix is short. This is load-bearing for the abstract’s latency comparison and should be quantified or the claim scoped more tightly to short-span erasures.
Sec. 4.1 parameterization: the eraser is a full trainable copy of the generator backbone (minus the LM head). Wall-clock latency in Figs. 3–4 includes the eraser forward pass and still favors KVEraser, but the ~2× parameter memory cost for hosting the eraser alongside the generator is not analyzed. For the “efficient” claim in production KV serving systems (the setting motivated in Sec. 1), either a memory/serving discussion or a lighter eraser architecture experiment is needed so that the Pareto comparison is not only latency-based.
Sec. 5.5 and Appendix E: on natural QA, ~49.5% of KVEraser failures remain corrupted-context reliance (answers that still use the distractor). Sec. 6 attributes the residual gap to full recompute mainly to training data scale/diversity, but the failure mix suggests that the local compensation interface (Eq. 4–5 intuition) is incomplete for a non-trivial subset of natural distractors. A short analysis by distractor position, semantic type, or distance from the query would better bound when local steering succeeds versus when suffix recomputation remains necessary, and would strengthen the transfer claim.

Circularity Check

0 steps flagged

No significant circularity: empirical ML method trained to match clean-prompt behavior and evaluated on held-out / unseen data.

full rationale

KVEraser is a standard supervised cache-editing method, not a first-principles derivation. The functional goal (Eq. 2) and the teacher-forced objective (Eq. 6) train the eraser so that decoding from the surrogate cache matches the frozen generator on the clean edited prompt; reported exact-match and latency are then measured on a held-out NIAH split and on three QA datasets never used in training (2WikiMultiHopQA, MuSiQue, IIRC). Eq. 5 is explicitly labeled mechanistic intuition and is not imposed as a constraint. Baselines (full recompute, delete-and-shift, instruction-only forgetting, local suffix repair) are independent procedures, not fitted renamings of the method. Sample filtering (keep only cases where the frozen model succeeds on the clean prompt) is ordinary supervision hygiene, not a self-definitional reduction of the reported metrics. No load-bearing uniqueness theorem, ansatz, or self-citation chain forces the quality–efficiency claims. The paper is self-contained against external benchmarks; residual gaps and single-span scope are limitations of generalization, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The central claim rests on standard transformer KV-cache mechanics, a functional (not tokenwise-exact) success criterion, and several engineering choices: eraser architecture as a frozen-generator backbone copy, conditioning only on prefix KV plus erased span, length-preserving replacement, and a two-stage synthetic-then-task training curriculum. Free parameters are ordinary training hyperparameters and fixed design knobs (span length, repair window size for baselines, learning rate). No new physical entities are postulated; the 'steering KV block' is an engineered module, not an independent natural object.

free parameters (4)

eraser learning rate / weight decay / epochs = lr=1e-5, wd=0.01, 1 epoch each stage
AdamW lr 1e-5, weight decay 0.01, 1 epoch pretrain and 1 epoch fine-tune; standard fitted training knobs that affect final quality.
erased-span length in data construction = ~100 tokens for Wikipedia spans
Pretraining and QA distractors use fixed ~100-token Wikipedia chunks; NIAH needles are short fixed templates. Results may depend on this length regime.
local suffix repair window fraction (baseline) = 15% of suffix
Baselines recompute 15% of the suffix; this hand-chosen fraction defines the strongest approximate baseline comparison.
pretrain/finetune sample counts and filtering = 80K pretrain; ~7.5K finetune
80K pretrain samples; ~7.5K fine-tune samples retained only when clean prompt succeeds and distractor flips the answer. Selection rules shape the learned behavior.

axioms (5)

domain assumption Causal self-attention makes each token's KV depend on its entire prefix, so editing an earlier span invalidates all later cached states for exact reuse.
Stated in Sec. 1–2 as the validity condition of KV reuse; standard transformer fact.
ad hoc to paper Success is functional behavioral match on high-probability responses under the edited prompt, not tokenwise reconstruction of the exact edited cache (Eq. 2).
Defines the paper's evaluation target; enables local steering without exact suffix rebuild.
ad hoc to paper A length-preserving steering block for the erased interval, conditioned on prefix KV and the erased span, can compensate residual suffix contamination for decoding.
Core modeling hypothesis in Sec. 4.1 / Eq. 4–5; not forced by transformer theory alone.
ad hoc to paper Single contiguous span deletion is the problem setting; multi-span deletion is deferred to iterative application.
Explicit scope limit in Sec. 2; load-bearing for claimed complexity and experiments.
domain assumption Teacher-forced NLL against clean-prompt continuations is a sufficient training objective for transferable erasing.
Standard supervised fine-tuning assumption applied to the eraser (Eq. 6).

invented entities (2)

KVEraser steering KV block (KV_steer) no independent evidence
purpose: Length-preserving learned replacement for the erased span's keys/values that steers future attention/decoding while reusing the contaminated suffix cache.
Engineered module introduced in Sec. 4; independent_evidence is false because its only validation is end-to-end task match inside this paper's training/eval loop.
Two-stage span-neighbor pretraining + erasing fine-tuning curriculum no independent evidence
purpose: Teach generic suppression of deleted spans then adapt to NIAH and factual-distractor QA without large annotated erase corpora.
Training recipe invented for this problem; not an external natural object.

pith-pipeline@v1.1.0-grok45 · 20479 in / 3725 out tokens · 31201 ms · 2026-07-14T17:47:46.227134+00:00 · methodology

0 comments

read the original abstract

Post-hoc context erasing over the KV cache is challenging because a local edit has a global consequence: once a span has been processed, its influence propagates into the cached states of all subsequent tokens. This issue arises naturally in long-context LLM applications, where stale retrieved facts, incorrect tool observations, retracted user preferences, or harmful prompt injections may be identified only after prefill. Exact erasing must then recompute all tokens after the deleted span, making its computational cost depend on suffix length rather than erased-span length. We introduce KVEraser, a learned KV-cache editing method for efficient localized context erasing. Given a processed context and a span to remove, KVEraser replaces only the KV states of the erased interval with learned steering states while reusing the remaining cache unchanged. To learn a transferable erasing mechanism, we build a two-stage training pipeline: generic span-neighbor pre-training teaches the eraser to suppress the influence of the erased span, while task-specific fine-tuning adapts this capability to downstream scenarios. Experiments show that KVEraser nearly matches full recomputation in post-erasure performance on in-domain tasks across 1K--32K context lengths, while its latency increases by only 24% compared with a 17.6x increase for full recomputation. KVEraser also generalizes to unseen long-document QA tasks with harmful factual distractors, achieving the best performance among approximate baselines with a 3--4x speedup over full recomputation.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Compute Globally, Materialize Locally: The Memory Contract of Sparse Event-KV
cs.AI 2026-07 conditional novelty 6.5

Source-omitted event KV rows can carry compact upstream state (semantic materialization); deliberate answer-free carriers raise recovery from 6% to 51% on Qwen3-8B, while passive natural mentions do not.

Reference graph

Works this paper leans on

12 extracted references · 3 linked inside Pith · cited by 1 Pith paper

[1]

PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling

Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Xiao Wen. PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling. arXiv preprint arXiv:2406.02069,

Pith/arXiv arXiv
[2]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

10 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186,

2019
[3]

IIRC: A Dataset of Incomplete Information Reading Comprehension Questions

James Ferguson, Matt Gardner, Hannaneh Hajishirzi, Tushar Khot, and Pradeep Dasigi. IIRC: A Dataset of Incomplete Information Reading Comprehension Questions. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1137–1147,

2020
[4]

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530,

Petko Georgiev et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530,

Pith/arXiv arXiv
[5]

Dense Passage Retrieval for Open-Domain Question Answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense Passage Retrieval for Open-Domain Question Answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781,

2020
[6]

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation.arXiv preprint arXiv:2510.07414,

11 Mufei Li, Dongqi Fu, Limei Wang, Si Zhang, Hanqing Zeng, Kaan Sancak, Ruizhong Qiu, Haoyu Wang, Xiaoxin He, Xavier Bresson, Yinglong Xia, Chonglin Sun, and Pan Li. Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation.arXiv preprint arXiv:2510.07414,

arXiv
[7]

KILT: a Benchmark for Knowledge Intensive Language Tasks

Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. KILT: a Benchmark for Knowledge Intensive Language Tasks. InProceedings of the 2021 Conference of the North American Chapter of the Associati...

2021
[8]

Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections.arXiv preprint arXiv:2510.26328,

David Schmotz, Sahar Abdelnabi, and Maksym Andriushchenko. Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections.arXiv preprint arXiv:2510.26328,

arXiv
[9]

Tools Fail: Detecting Silent Errors in Faulty Tools

Jimin Sun, So Yeon Min, Yingshan Chang, and Yonatan Bisk. Tools Fail: Detecting Silent Errors in Faulty Tools. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14272–14289,

2024
[10]

Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025a

An Yang et al. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025a. 12 Jingbo Yang, Bairu Hou, Wei Wei, Yujia Bao, and Shiyu Chang. KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems,

Pith/arXiv arXiv
[11]

APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding

Xinyu Yang, Tianqi Chen, and Beidi Chen. APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding. InInternational Conference on Learning Representations, 2025b. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A Dataset for Diverse, Explainable Multi-...

2018
[12]

Data construction for QA.We use the 2019-08-01 Wikipedia snapshot

13 Appendix A Additional details for pre-training Data construction for NIAH.Same as (Hsieh et al., 2024), we use sampled Paul Graham’s essays for irrelevant background text. Data construction for QA.We use the 2019-08-01 Wikipedia snapshot. Inspired by the success of hard negative mining in training dense retrievers (Karpukhin et al., 2020), we construct...

2024

[1] [1]

PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling

Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Xiao Wen. PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling. arXiv preprint arXiv:2406.02069,

Pith/arXiv arXiv

[2] [2]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

10 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186,

2019

[3] [3]

IIRC: A Dataset of Incomplete Information Reading Comprehension Questions

James Ferguson, Matt Gardner, Hannaneh Hajishirzi, Tushar Khot, and Pradeep Dasigi. IIRC: A Dataset of Incomplete Information Reading Comprehension Questions. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1137–1147,

2020

[4] [4]

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530,

Petko Georgiev et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530,

Pith/arXiv arXiv

[5] [5]

Dense Passage Retrieval for Open-Domain Question Answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense Passage Retrieval for Open-Domain Question Answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781,

2020

[6] [6]

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation.arXiv preprint arXiv:2510.07414,

11 Mufei Li, Dongqi Fu, Limei Wang, Si Zhang, Hanqing Zeng, Kaan Sancak, Ruizhong Qiu, Haoyu Wang, Xiaoxin He, Xavier Bresson, Yinglong Xia, Chonglin Sun, and Pan Li. Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation.arXiv preprint arXiv:2510.07414,

arXiv

[7] [7]

KILT: a Benchmark for Knowledge Intensive Language Tasks

Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. KILT: a Benchmark for Knowledge Intensive Language Tasks. InProceedings of the 2021 Conference of the North American Chapter of the Associati...

2021

[8] [8]

Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections.arXiv preprint arXiv:2510.26328,

David Schmotz, Sahar Abdelnabi, and Maksym Andriushchenko. Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections.arXiv preprint arXiv:2510.26328,

arXiv

[9] [9]

Tools Fail: Detecting Silent Errors in Faulty Tools

Jimin Sun, So Yeon Min, Yingshan Chang, and Yonatan Bisk. Tools Fail: Detecting Silent Errors in Faulty Tools. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14272–14289,

2024

[10] [10]

Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025a

An Yang et al. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025a. 12 Jingbo Yang, Bairu Hou, Wei Wei, Yujia Bao, and Shiyu Chang. KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems,

Pith/arXiv arXiv

[11] [11]

APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding

Xinyu Yang, Tianqi Chen, and Beidi Chen. APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding. InInternational Conference on Learning Representations, 2025b. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A Dataset for Diverse, Explainable Multi-...

2018

[12] [12]

Data construction for QA.We use the 2019-08-01 Wikipedia snapshot

13 Appendix A Additional details for pre-training Data construction for NIAH.Same as (Hsieh et al., 2024), we use sampled Paul Graham’s essays for irrelevant background text. Data construction for QA.We use the 2019-08-01 Wikipedia snapshot. Inspired by the success of hard negative mining in training dense retrievers (Karpukhin et al., 2020), we construct...

2024