{"id":"36f5e0a1-84a1-4f23-a959-0de3dea3be89","arxiv_id":"2607.28069","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Offline LoRA Writer distillation produces native position-independent document KV caches that approach full-recompute quality while keeping the pretrained Reader and cache-hit path unchanged.","lead":"SemPIC trains a LoRA Writer offline so independently cached document KVs work under new queries and document orders, without changing the decoder at serve time. It lifts mean micro-F1 from 0.53 (KV Packet) to 0.60 versus 0.62 full recompute across three models and four tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The headline mean F1 gain rests on domain-matched Writers and is not shown to hold under the paper's own PIC premise of unseen compositions without mixture training.","rationale":"The Reader correctly flags that KD sufficiency under unseen compositions is the soft spot and that attention is only descriptive, and CONDITIONAL is the right tier given missing artifacts and the MuSiQue regression. The sharper load-bearing issue is narrower: Table 1's 0.53\to0.60 average is produced under domain-matched Writers, whereas the paper's PIC motivation and transfer section show that the same method needs mixture (or in-domain) training to stay near that operating point. That does not overturn the systems contribution—native-KV Writer, unchanged Reader, KV Gradient Checkpointing, and strong in-domain results remain—but it means the abstract-level claim overstates what is demonstrated for truly position- and domain-independent reuse. I therefore keep CONDITIONAL rather than REJECT, with the concrete mixture-only re-eval as the decisive check. Agreement with the Reader is partial because they located the right premise (generalization of offline KD) but under-weighted the domain-matched vs. cross-composition gap already visible in the paper's tables.","tokens_in":15548,"tokens_out":689,"duration_ms":15474,"concrete_test":"Retrain one fixed Writer per model on the Mixture split only; evaluate zero-shot on the same 100-example held-out cells as Table 1 with no per-task adapters. If the 12-cell mean micro-F1 falls more than ~0.03 below 0.60 or loses the 10/12 wins over Packet, the headline gain does not transfer to the paper's stated PIC setting and should be restated as domain-adapted compilation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (Abstract; Table 1) is that a LoRA Writer trained by behavioral KD (Eq. 8) into native KVs raises mean micro-F1 from 0.53 (KV Packet) to 0.60, approaching Full Recompute (0.62), while preserving an unchanged Reader and cache-hit path. That claim is load-bearing on the premise that offline, document-only adaptation closes contextual incompleteness for later unseen compositions (Sections 1, 2, 5). Table 1, however, reports per-model-and-domain Writers (Setup: \"trained per model and domain\"). Under the paper's own cross-domain protocol (Table 2; Supp. Tables 5–6), single-source Writers often lag Packet on out-of-domain targets and only approach the headline regime under Mixture training (Qwen3-8B averages: Packet 0.51, SemPIC/Joint 0.57). The Llama-3.1-8B MuSiQue in-domain cell also regresses (0.28 vs Packet 0.37). Thus the 0.53\to0.60 lift is not yet established as a property of semantic PIC under the motivating reuse setting (changing instructions, histories, orders, and document sets) without domain-matched or mixture supervision. The attention diagnostics are explicitly non-causal (Sections 4–5), so they do not independently secure the generalization claim.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper addresses contextual incompleteness in position-independent KV caching (PIC): independently compiled document KVs are position-correctable via RoPE re-rotation but lack the future context in which they will be consumed. Motivating diagnostics on KV Packet show strong reduction of near-boundary attention deviation but persistent interior and task-level residuals. SemPIC trains a LoRA-enabled Writer to compile native per-layer document KVs by behavioral distillation against full-recompute teacher logits, while freezing the pretrained decoder as an unchanged Reader with adapters off at read time. Adaptation is offline; cache-hit serving retains the standard KV interface. KV Gradient Checkpointing keeps the differentiable KV path while discarding Writer internals. On Llama-3.1-8B, Qwen3-4B, and Qwen3-8B across Biography, HotpotQA, MuSiQue, and NIAH, SemPIC raises mean micro-F1 from 0.53 (KV Packet) to 0.60 versus Full Recompute at 0.62, with lower Full-relative interior attention error in all 12 settings.","tokens_in":15954,"tokens_out":1406,"duration_ms":42873,"significance":"If the result holds under the stated PIC reuse regime, the work is a meaningful systems/ML contribution: it shows that offline, document-wide adaptation of native KVs can approach full recompute quality without changing the online Reader or cache-hit path, unlike online selective recompute or auxiliary cache interfaces. The boundary–interior diagnostic is a useful design probe; KV Gradient Checkpointing is a practical enabler for all-layer KV-mediated training. Strengths include a clean Writer–Reader split, preservation of the standard serving interface, multi-model multi-task evaluation, efficiency tradeoff plots against selective recompute, and honest labeling of attention analyses as descriptive rather than causal.","major_comments":[{"comment":"Abstract and §6 (Table 1) headline the 0.53→0.60 mean micro-F1 lift, but Setup states adapters are “trained per model and domain.” That matches in-domain held-out splits, not the stronger PIC premise of arbitrary reuse across changing document sets/domains without matched supervision. Cross-domain results (Table 2; Supp. Tables 5–6) show single-source Writers often lag Packet on OOD targets and only approach the headline regime under Mixture training (e.g., Qwen3-8B averages 0.51 Packet vs 0.57 SemPIC/Joint). Either narrow the abstract claim to domain-matched Writers, or promote mixture/cross-composition results as primary evidence for the motivating reuse setting.","section":"Abstract; §6 Setup; Table 1; Table 2"},{"comment":"Table 1, Llama-3.1-8B MuSiQue: SemPIC drops to 0.28 vs KV Packet 0.37 and Full 0.39, while Joint recovers to 0.38. This is the clearest in-domain failure of document-wide adaptation alone. The paper notes Joint complementarity but does not analyze when interior adaptation hurts multi-hop composition. A load-bearing claim that SemPIC “approaches Full Recompute” needs either failure-mode analysis (document count, hop structure, cache linking) or explicit scope limits on multi-hop tasks where boundary states remain necessary.","section":"Table 1; §6 Semantic Compilation Recovers PIC Quality"},{"comment":"Each model–task–method cell uses n=100 with no error bars, bootstrap intervals, or significance tests (Setup; Table 1). Several gains are small (e.g., Llama HotpotQA 0.44→0.45; Qwen3-4B HotpotQA 0.29=0.29). The overall mean is sensitive to Biography/NIAH and to the MuSiQue regression. For a central quality claim against Full Recompute and Packet, report variability or repeated seeds; otherwise temper “approaching Full Recompute” language to match the precision of the evidence.","section":"§6 Setup; Table 1"},{"comment":"§4–5 and Figure 5 correctly state that lower R_int is descriptive and does not establish that interior attention error causes F1 gains. The design motivation still leans on this asymmetry (“motivating adaptation of the document representation itself”). Keep the diagnostic, but avoid implying mechanism in the contribution list and conclusion unless an ablation (e.g., interior-only vs full-document LoRA, or attention-regularized KD) ties representation change to task recovery.","section":"§4; §5; Figure 5; §7"}],"minor_comments":[{"comment":"Eq. (8) uses τ=1 and summed token-level KL; briefly justify against the more common temperature>1 KD setup, or note sensitivity if any was checked.","section":"§5 Behavioral supervision; Eq. (8)"},{"comment":"Figure 3/5 recovery bars are capped or hatched differently; unclipped values in text help, but a single consistent caption convention would reduce misreading of values >1.","section":"Figure 3; Figure 5"},{"comment":"Related work cites concurrent arXiv items (KV Packet, COMB, C2KV) appropriately; ensure camera-ready versions and date stamps remain accurate relative to this submission.","section":"§3"},{"comment":"Supplementary memory protocol notes two-second nvidia-smi sampling and single probes; state this limitation next to Table 3 in the main text so peak-memory claims are not over-read as full training profiles.","section":"Table 3; Supp. B"},{"comment":"Minor typos/spacing in the abstract opening (“Long-contextretrievalandagenticworkloads”) suggest PDF text extraction or kerning issues; clean for production.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"Solid systems-oriented ML paper with a clean interface story. The main risk is abstract overclaim relative to domain-matched training and one clear multi-hop regression. I would not reject: the method and checkpointing trick are publishable after claim recalibration and tighter statistics/failure analysis. Fit is appropriate for a venue that takes LLM serving + representation learning; less so if the venue expects causal mechanism proofs for every diagnostic."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The real move here is the adaptation locus. Instead of online repair or boundary tokens, they train a LoRA Writer that emits ordinary per-layer document KVs for a frozen Reader, keep the cache-hit path untouched, and add KV Gradient Checkpointing so the all-layer KV interface stays on the gradient path without OOMing. That is a clean systems idea, and the design-space framing against CacheBlend/EPIC, KV Packet, and COMB is honest.\n\nWhat works: Table 1 is the right experiment. On three models and four tasks, domain-matched SemPIC lifts mean micro-F1 from Packet’s 0.53 to 0.60 vs Full 0.62, wins 10/12 cells, and Joint shows boundary + document-wide can help. Interior Full-relative attention error drops in every setting, with explicit non-causal language. Checkpointing is practical: naive OOM on longer tasks, checkpointed path fits, and serving is unchanged. Citations and math are standard and solid enough for this venue class.\n\nSoft spots, in proportion. The stress-test is mostly right: Setup trains per model and domain, so the abstract mean is not yet “unseen compositions” in the strong PIC sense. Cross-domain tables need mixture training to approach the headline regime; single-source Writers often lag Packet OOD. Llama MuSiQue regresses in-domain (0.28 vs 0.37). n=100, no error bars, no code/data. Those are real limits on the generalization claim, not reasons to dismiss the method. Attention diagnostics support the motivation; they do not prove mechanism.\n\nThis is for people building RAG/agent KV reuse stacks who care about recomputation-free hits and native interfaces. Worth a serious referee. I’d engage, cite the Writer/checkpointing split when discussing PIC loci, and push for mixture-default numbers plus the MuSiQue failure case. Send to review.","headline":"Solid systems paper: native-KV Writer + checkpointing beats boundary PIC on matched domains and nearly matches full recompute, but the headline 0.53→0.60 lift is domain-matched and weakens under the paper’s own cross-domain PIC premise.","tokens_in":16564,"tokens_out":501,"would_cite":true,"duration_ms":10851,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Training a document Writer offline can make independently cached KV states behave like full-context recomputation for reuse under new layouts.","keywords":["position-independent caching","KV cache reuse","behavioral distillation","LoRA Writer","long-context RAG","KV gradient checkpointing","cache-hit decoding"],"falsifier":"On held-out multi-document layouts the paper already studies, if SemPIC’s cache-hit micro-F1 stayed near the no-recompute or boundary-only baseline instead of approaching full recompute—especially on tasks where interior attention error did not fall—the central claim would fail.","tokens_in":16434,"feed_emoji":"🧩","tokens_out":919,"duration_ms":20838,"temperature":0.7,"pith_summary":"Long-context systems keep reusing the same documents under new instructions, histories, and orders, but ordinary prefix caches only help when the shared text sits behind an identical prefix. Position-independent caches try to prepare each document once and stitch it in later, yet those caches were built without the surrounding context they will later see, so quality often collapses. The paper shows that fixing only the block boundaries helps near the edges but leaves interior mismatch and task error. SemPIC instead trains a LoRA-enabled Writer offline to compile each document into the model’s native per-layer key/value caches so a frozen Reader, using the ordinary cache-hit path, nearly matches full recomputation. Across three models and four tasks, mean micro-F1 rises from 0.53 (boundary-only baseline) to 0.60, near full recompute at 0.62, without changing online decoding.","feed_headline":"Offline-trained document caches nearly match full recompute","feed_subtitle":"A LoRA Writer builds native KV caches so frozen models reuse documents under new orders without recomputing them","key_machinery":"SemPIC’s Writer–Reader split through the native KV interface: LoRA is on only while the Writer compiles each document into standard per-layer K/V tensors; the frozen Reader consumes those caches with LoRA off and is trained by KL distillation to the full-context teacher. KV Gradient Checkpointing keeps those cached KVs differentiable while recomputing Writer internals on the backward pass.","core_discovery":"Independently compiled document KV caches can be made semantic—preserving downstream predictive behavior under unseen prefixes and compositions—by offline behavioral distillation of a LoRA Writer that produces native per-layer KVs for an unchanged pretrained Reader, lifting mean micro-F1 from 0.53 (KV Packet) to 0.60 versus 0.62 for full recompute.","pith_inferences":["If Writer-compiled caches transfer across domains as the cross-domain tables suggest, shared document stores could host one semantic cache per document rather than per request layout.","The remaining first-token attention peak inside each block hints that block-local artifacts may still need explicit treatment even when task F1 recovers.","Agent memory and tool-result stores that repeatedly inject the same chunks under new plans are a natural next stress test beyond the four retrieval-style tasks.","Because the Reader never changes, the method could sit under existing serving stacks that already relocate and re-rotate standard KV caches."],"forward_implications":["Reusable document KVs can be compiled once offline and reused across changing queries, histories, and document orders without online document recomputation.","Cache-hit serving keeps the standard KV interface and pretrained decoder path; adaptation cost is amortized at construction time.","Document-wide Writer adaptation and learned boundary states can be co-trained and are complementary on several settings.","KV Gradient Checkpointing makes all-layer differentiable Writer training feasible on long documents that otherwise run out of memory.","Deployments that forbid online selective recompute gain a quality path close to full prefill while staying on the recomputation-free cache-hit path."],"fun_headline_variants":["LoRA Writer builds semantic KV caches that nearly match full recompute","Offline distillation lifts document KV F1 from 0.53 to 0.60","SemPIC compiles native per-layer KVs for unchanged pretrained readers","Position-independent caches approach full recompute via behavioral distillation","Adapted document KVs let frozen models reuse text under new orders"],"cache_read_input_tokens":128,"weakest_assumption_plain":"Matching the full-context model’s output token distributions offline is enough to fix the missing future context inside independently built document caches when they are later mixed in new orders and queries.","fun_headline_variants_meta":{"raw":{"variants":["LoRA Writer builds semantic KV caches that nearly match full recompute","Offline distillation lifts document KV F1 from 0.53 to 0.60","SemPIC compiles native per-layer KVs for unchanged pretrained readers","Position-independent caches approach full recompute via behavioral distillation","Adapted document KVs let frozen models reuse text under new orders"]},"model":"grok-4.5","effort":"low","cost_usd":0.003719,"raw_usage":{"total_tokens":1179,"prompt_tokens":730,"num_sources_used":0,"completion_tokens":94,"cost_in_usd_ticks":37188000,"prompt_tokens_details":{"text_tokens":730,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":355,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":730,"tokens_out":94,"duration_ms":7340,"temperature":1.0,"reasoning_tokens":355,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T18:47:50.219174+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On held-out multi-document layouts the paper already studies, if SemPIC’s cache-hit micro-F1 stayed near the no-recompute or boundary-only baseline instead of approaching full recompute—especially on tasks where interior attention error did not fall—the central claim would fail.","supporting_citations":[],"review_version":1}