{"id":"2142cbe7-57ef-47c5-adbe-d8e39335627d","arxiv_id":"2608.00311","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"SeDeM stores long contexts as compact query-independent memory blocks, selects only query-relevant blocks, and decompresses them into intermediate decoder hidden states, beating compression baselines on four long-context QA benchmarks.","lead":"Long-context QA models pay quadratic attention and often miss the evidence. SeDeM compresses documents into small memory blocks, decompresses only the blocks a question needs into a middle LLM layer, and reports higher quality than full-context fine-tuning on three of four benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gains may come from evidence selection, not decompression: the ablations remove selection and decompression together, and no control decodes from the raw tokens of the same top-K segments.","rationale":"The reader's weakest assumption was that decompressed hidden states remain compatible with native decoder-layer states, citing Appendix A.13's Lipschitz caveat. That is a real theoretical risk, but the paper's Stage 2 training directly optimizes decoder continuation on injected states, so the empirical results can hold even if the Lipschitz argument is imperfect. My concern is more directly tied to the central claim: the controlled comparison cannot separate the contribution of decompression from the contribution of top-K evidence selection. Section 7.1 compares decompressed reconstructions vs. direct memory conditioning, but both arms are selector-disabled and condition on all blocks; therefore the comparison changes both the representation and the selection regime at once. Meanwhile, Section 7.2 shows selection budget alone moves scores by several F1 points. Without a raw-selected-context or original-hidden-state control at the same K, the abstract's attribution of the gains to 'expanding selected memories into decoder-compatible hidden states' is not supported. This is a concrete, fixable experimental gap rather than a fatal flaw, so it reinforces the conditional verdict rather than changing it. I agree with the reader that injection compatibility is the natural theoretical soft spot, but the missing control is the load-bearing empirical soft spot.","tokens_in":22941,"tokens_out":7525,"duration_ms":82966,"concrete_test":"Run Table 2 with two additional control arms at the same selection budget K and same evaluation pipeline: (a) Raw-selected control: map SEDEM's test-time top-K block indices back to their original token segments and decode from those raw tokens through the same LoRA-adapted Llama-3.2-1B/3B, with no compression and no injection; (b) Hidden-selected control: take the original layer-ℓ_extract hidden states of those same selected segments and inject them at layer ℓ_inject (projected to decoder dimension), bypassing mean-pooling and the MLP decompressor. If either control reaches or exceeds SEDEM's F1 on 2WikiMultiHopQA, MuSiQue, QASPER, or HotpotQA-Distractor at equal K, then selective decompression is not the source of the quality advantage. Report per-dataset F1/ROUGE-L with K fixed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Concern: the empirical design does not isolate the proposed selective-decompression mechanism from the effect of evidence selection, so the headline claim that gains 'come from expanding selected memories into decoder-compatible hidden states rather than from compression alone' (Section 7.1) is not established. The no-decompression ablation in Table 4 is run selector-disabled with all segment blocks, removing selection and decompression simultaneously. Table 5 shows selection budget alone changes HotpotQA-Distractor F1 from 50.93 (K=2) to 45.65 (K=10), so selection is a large effect. The paper never compares SEDEM against decoding from the raw tokens (or the original extracted hidden states) of the same top-K selected segments. Absent that control, the margin over ICAE/500x could come from the selector's access to gold evidence labels plus a larger effective decoder budget (up to 128K positions, e.g., ~1104 tokens for QASPER at K=8, Table 8), not from hidden-state decompression. Appendix A.13's Lipschitz argument is explicitly motivational and rests on a 'strong assumption,' so it does not fill this gap. A matched selected-raw-context control is needed to attribute the reported gains to decompression.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SeDeM, a long-context QA framework that decouples compact memory storage from decoder conditioning. A frozen LLM encoder extracts hidden states at an intermediate layer; each context segment is compressed by local mean pooling and a shared projection into memory blocks; a query-conditioned late-interaction selector chooses top-K blocks; and a learned decompressor expands only the selected blocks into hidden states injected at an intermediate decoder layer, after which the decoder continues to generate. Training is two-stage: Stage 1 reconstructs compressed/decompressed states against projected encoder states with next-token and distillation losses, and Stage 2 adds evidence-supervised selection, QA loss, and LoRA adaptation. Experiments on 2WikiMultiHopQA, MuSiQue, QASPER, and HotpotQA-Distractor with Llama-3.2-1B/3B report higher F1/ROUGE-L than the reimplemented compression baselines, and with the 3B backbone, SeDeM exceeds full-context fine-tuning on three datasets. The paper also reports lower time-to-first-token and improved throughput relative to ICAE, plus ablations on decompression, selection budget, layer depth, granularity, and transfer.","tokens_in":23291,"tokens_out":5686,"duration_ms":58896,"significance":"If the central result holds, SeDeM makes a useful architectural contribution: query-independent compact memories can be reused across queries, storage is decoupled from decoder conditioning, and a smaller encoder can serve a larger decoder. The paper is transparent about its limitations, provides extensive baseline reproduction details in Appendix A.8, includes parameter/FLOP accounting, and openly frames Appendix A.13's Lipschitz argument as motivational rather than as proof. The efficiency numbers and cross-model transfer are interesting. However, the headline attribution of the QA gains to selective decompression is not yet established by the current experiments, and the comparison to baselines is confounded by unequal decoder-side budgets and by the selector's access to gold evidence labels during training.","major_comments":[{"comment":"The main comparison does not isolate selective decompression from evidence selection. Table 4 ablates decompression only in a selector-disabled, all-blocks setting and uses a different ('internal non-final extraction-layer') configuration than the main model in Table 2; Table 5 shows that changing the selection budget K alone moves HotpotQA-Distractor F1 by about 5 points. Since the Table 2 SEDEM rows use top-K selection supervised by gold evidence labels and the compression baselines do not select evidence, the reported margin over ICAE/500x could come largely from the selector and from the larger effective decoder budget (Table 8) rather than from hidden-state decompression. A matched control is needed: decode from the raw tokens (or raw extracted hidden states) of the same top-K selected segments chosen by the SeDeM selector, with the same decoder and LoRA configuration, and compare a","section":"§6.1, Tables 2 and 5; §7.1, Table 4"},{"comment":"The compression budgets are not matched across methods. 500xCompressor is globally truncated at 500 tokens (A.8.5), ICAE conditions on 128 memory tokens, and SeDeM decodes from 128K positions, with K=8 giving about 1104 decoder-side tokens on QASPER (Table 8). The claim of 'higher QA scores than the evaluated compression baselines' is therefore a comparison across different effective context budgets as well as different mechanisms. The authors should either equalize the decoder-side budget/number of conditioning positions or present a budget-matched variant (e.g., ICAE and 500x with the same selected K segments) so that the marginal benefit of decompression is separated from the benefit of more conditioning tokens.","section":"§5 vs Appendix A.8.5 and Table 8"},{"comment":"All headline numbers are single-run point estimates. The claims of surpassing full-context fine-tuning on three datasets and of ranking above all compression baselines rest on F1 differences that in some cases are only a few points (e.g., QASPER 26.74 vs 23.44; HotpotQA 58.30 vs 45.09). Without multiple seeds or error bars, run-to-run variance could change the ordering. Report means and standard deviations over at least three seeds for SeDeM and for the closest baselines on the main comparisons, or provide a sensitivity analysis for the top three rows of Table 2.","section":"Table 2 and §6.1"},{"comment":"The decompression ablation's reference row is not the main SeDeM configuration: the caption states that 'the SEDEM reference in this controlled ablation uses the internal non-final extraction-layer configuration; Table 2 reports the designated main configuration.' This makes the ablation internally controlled, but it does not directly support the main-table claim that selective decompression is the source of the gains in Table 2. Either re-run the Table 4 comparison with the exact main configuration, or explicitly state in §7.1 that the ablation is a configuration-level diagnostic and not a component analysis of the reported main results.","section":"Table 4 caption"}],"minor_comments":[{"comment":"The '—' entries for Activation Beacon ROUGE-L are explained as 'a corrected evaluation for which the paired corrected ROUGE-L value was not available.' Please clarify what 'corrected evaluation' refers to and why only F1 is re-verified; as written, the footnote is hard to interpret.","section":"Table 2 footnote"},{"comment":"Figure 1 contains rendering artifacts in the submitted text (placeholder strings such as '/gid00001/gid00015'); the figure needs to be regenerated with the proper labels and arrowheads.","section":"Figure 1"},{"comment":"The name is spelled 'SeDeM' in the abstract and running title but 'SEDEM' throughout the body and tables/appendices. Standardize the capitalization.","section":"Abstract and body"},{"comment":"The paragraph beginning 'Block-structured memories support selection' is a one-sentence standalone lead-in before the learnable-query compressor discussion; it reads as a heading rather than a result. Either move this sentence into the following paragraph or add a concrete comparison there.","section":"§7.2"},{"comment":"The stability bound is clearly labeled as motivational, which is appropriate. However, the text would benefit from one explicit sentence stating that the bound does not constrain the trained decompressor in this paper and that the empirical losses in Eqs. (9)–(11) are the actual mechanism for state matching.","section":"Appendix A.13"}],"recommendation":"major_revision","confidential_remarks":"This is a well-structured and unusually transparent paper: the limitations section is honest, the baseline reproduction details are thorough, and the efficiency accounting is a strength. The main issue is a missing control that would let the authors attribute the empirical gains to selective decompression rather than to evidence selection and a larger decoder budget. I believe this is fixable with additional experiments (raw-token selected-context control, budget-matched baselines, and repeated-run variance estimates) and that the paper does not need to be rejected. No concerns about citation practices; HMT is reimplemented as a baseline despite shared authorship, which is disclosed and handled reasonably."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. SeDeM's architecture is a genuine combination: query-independent pooling of hidden states into segment blocks, ColBERT-style late-interaction for top-K block selection, and an MLP that decompresses only the chosen blocks into hidden states injected at an intermediate decoder layer. The decoupling of storage from decoder conditioning is the right thing to test, and the paper tests it over four QA benchmarks with 1B and 3B backbones. The system-level tables are coherent, the baseline reimplementations are documented in detail, and the limitations section is honest—it flags MuSiQue, the empirical layer choices, and the limited RULER check.\n\nThe main problem is that the paper's central mechanistic claim is not supported by its ablations. Section 7.1 says the gains come from selective decompression rather than compression alone, but Table 4 removes selection and decompression together: the 'no decomp' variants are selector-disabled and condition on all segment blocks. Table 5 shows selection budget is a large factor on its own (50.93 to 45.65 F1 on Hotpot as K rises). There is no control that runs the same top-K selected evidence through the raw-token decoder or through direct memory conditioning. Without that, the margin over ICAE could be largely the supervised selector plus a bigger effective decoder budget, not the decompressor. That is fixable: add a matched selected-raw-context control.\n\nOther issues are minor or moderate. Headline numbers are single-run point estimates with no error bars. The baselines use different compression budgets—500x is globally truncated, ICAE's memory size is not matched. No code released. None of these wreck the paper; they limit how strongly we can cite the numbers. The stability derivation in A.13 is explicitly motivational and correctly hedged. There is no fitting-labeled-as-prediction in the empirical section.\n\nWho this is for: people working on long-context inference, compression, and retrieval-style conditioning. The system is plausible and worth testing. It deserves peer review, not a desk reject, but reviewers should ask for the selection/decompression control and for artifacts. My recommendation: engage with it, treat the system result as provisional, and treat the mechanism claim as unproven until the control exists.","headline":"A real architectural idea with coherent system results, but the mechanism claim is confounded: selection and decompression are ablated together.","tokens_in":23729,"tokens_out":4205,"would_cite":true,"duration_ms":40626,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SeDeM claims that long-context question answering can be made both cheaper and more accurate by decoding from selected, decompressed hidden-state memories instead of from the full context or from compressed memory tokens.","keywords":["long-context QA","context compression","hidden-state memory","selective decompression","top-k selection","memory blocks","intermediate-layer injection","efficient inference"],"falsifier":"Measure the actual per-state reconstruction gap between decompressed hidden states and the decoder's own hidden states for the same selected segments, or run an oracle experiment that replaces the decompressor with the true full-context hidden states for the selected segments. If QA accuracy does not improve substantially under oracle injection, selection is the real bottleneck; if accuracy still falls short of full-context quality, reconstruction error at the injection layer is the bottleneck.","tokens_in":22868,"feed_emoji":"⚡","tokens_out":5751,"duration_ms":52402,"temperature":0.7,"pith_summary":"SeDeM argues that the failure point in soft context compression is not storage but conditioning: forcing a decoder to generate from highly compressed memory tokens. The method stores a long context as compact per-segment blocks of hidden states, selects the blocks most relevant to the query, and expands only those blocks into hidden states injected at a middle decoder layer. On four long-context question-answering benchmarks, the paper reports that this beats all evaluated compression baselines at both 1B and 3B model scale, and with the 3B backbone it exceeds full-context fine-tuning on three of the four datasets. It also reports lower time-to-first-token and higher decoding throughput than direct memory-token conditioning. The central claim is that selective decompression recovers most of the benefit of full-context reasoning while spending compute proportional to the selection budget rather than the context length.","feed_headline":"SeDeM beats full-context fine-tuning on 3 of 4 long-context QA tests","feed_subtitle":"It stores context as compact blocks, decompresses only query-relevant ones, and cuts time-to-first-token.","key_machinery":"The central object is a memory bank of segment-level hidden-state blocks with a separate expansion pathway. Each context segment is encoded once to an intermediate layer, collapsed by non-overlapping mean pooling plus a learned projection into a small number of memory vectors per segment. A ColBERT-style late-interaction scorer (max over slots, sum over query tokens, averaged over heads) ranks blocks, and a shared two-layer MLP reshapes each selected memory vector into multiple decoder-dimension states for injection at an intermediate decoder layer. Because compression is done by pooling rather than by learned attention tokens, memory construction is query-independent and parallelizable; bec","core_discovery":"The central claim is that decoupling compact memory storage from decoder conditioning works better than making compressed memories double as decoder input. The paper constructs a query-independent memory bank by running a frozen encoder over context segments, mean-pooling local chunks of intermediate-layer hidden states, and projecting them into compact memory blocks. A late-interaction selector scores blocks against the query and keeps the top K; an MLP decompressor expands each selected memory vector back into several decoder-dimension hidden states and injects them at an intermediate decoder layer, after which a frozen decoder with lightweight adapters generates the answer. The reported r","pith_inferences":["A natural untested extension the paper leaves implicit is adaptive per-query K: letting selector confidence or answer likelihood decide how many blocks to expand instead of using a fixed budget.","Because the memory bank is query-independent and the selector is a light scoring head, the architecture resembles latent retrieval; precomputing banks once for static corpora and running only selection and decompression at query time would make the reported efficiency gains larger.","The stability argument in the appendix suggests a diagnostic before adopting the method on a new backbone: measure the distributional gap between decompressed and genuinely processed hidden states at the injection layer, and if it is large, tune the injection depth or increase reconstruction weight first.","The paper's own limitation that the 3B model stays below full-context fine-tuning on one multi-hop dataset suggests the method currently favors settings where selected evidence is sufficient; a hybrid that falls back to full context when selector confidence is low could broaden the method's reach."],"forward_implications":["Decoder compute during generation scales with the number of selected blocks K rather than with the full context length N.","The same query-independent memory bank can answer multiple queries over the same context without re-running context compression.","The selection budget K acts as a tunable quality-efficiency trade-off: small budgets help single-hop or distractor-heavy tasks, while larger budgets help multi-hop tasks before saturating.","Removing the decompression step and feeding memory tokens directly to the decoder is substantially weaker, implying the expansion step carries most of the benefit.","A smaller encoder can build memories that a larger decoder consumes, separating context-processing cost from generation capacity."],"fun_headline_variants":["SeDeM stores context in compact blocks, decompresses only query-relevant ones","SeDeM: query-selected decompression beats full-context fine-tuning on 3/4 sets","SeDeM cuts long-context cost: only decompress memory blocks the query triggers","SeDeM: decouple memory from decoder, select relevant blocks, beat full-context FT"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the decompressor's outputs land close enough to the decoder's native hidden-state distribution at the injection layer that the upper decoder layers behave as if they had processed the selected context directly; the paper's own Appendix A.13 explicitly calls the local Lipschitz condition supporting this a strong assumption for Transformers, so if injected states are out-of-distribution, accuracy degrades no matter how good the selector is.","fun_headline_variants_meta":{"raw":{"variants":["SeDeM stores context in compact blocks, decompresses only query-relevant ones","SeDeM: query-selected decompression beats full-context fine-tuning on 3/4 sets","SeDeM cuts long-context cost: only decompress memory blocks the query triggers","SeDeM: decouple memory from decoder, select relevant blocks, beat full-context FT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000591,"raw_usage":{"total_tokens":2617,"prompt_tokens":760,"completion_tokens":1857,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":1761}},"tokens_in":504,"tokens_out":1857,"duration_ms":10495,"temperature":1.0,"reasoning_tokens":1761,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:44:36.638516+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the actual per-state reconstruction gap between decompressed hidden states and the decoder's own hidden states for the same selected segments, or run an oracle experiment that replaces the decompressor with the true full-context hidden states for the selected segments. If QA accuracy does not improve substantially under oracle injection, selection is the real bottleneck; if accuracy still falls short of full-context quality, reconstruction error at the injection layer is the bottleneck.","supporting_citations":[],"review_version":1}