{"id":"6ecec17b-6694-40fb-b6ad-ea9a58faefeb","arxiv_id":"2505.19812","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"EMLoC prunes and compresses demonstration examples in multimodal long contexts, reducing inference cost up to 77% while matching or slightly beating full-context accuracy.","lead":"This paper describes EMLoC, a method that lets multimodal AI models learn from many examples at inference time without any training, by compressing the examples into a small memory. It cuts the input length by about 77% while keeping accuracy similar, which makes long-context adaptation cheaper and faster.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 77% context-length reduction compares an uncompressed prompt length (MLoC) to a pruned KV-memory size (EMLoC), so the headline number conflates KV-cache compression with true context-length reduction and needs a precise metric definition.","rationale":"The reader's verdict of CONDITIONAL is appropriate, and the JS-divergence proxy is indeed the methodological weakest point: the pruning decision (Eq. 8) and the per-step JS check (Section 3.2) assume that preserving answer-token output distributions preserves task accuracy, which is only an empirical proxy. However, I judge the metric-definition issue to be the single most load-bearing concern for the central claim as written, because the headline number '77% context-length reduction' is the paper's main quantitative result. If the 2600 figure counts retained KV tokens while the 11338 figure counts raw prompt tokens, the two are not commensurable; the actual reduction in tokens processed during adaptation is much smaller. This does not suggest fraud; it suggests imprecision in what is being measured. The paper also has internal evidence relevant to this: Table 9 shows EMLoC's adaptation time is 144s versus MLoC's 28s, so the efficiency gain is not in the adaptation phase but in later inference on the compressed memory. On the accuracy side, the YouCook2 result (102.0 vs MLoC's 108.8) is the only place where the paper explicitly reports a large degradation, and the conclusion glosses over it. The theoretical bound (Eq. 16) has the additional problem that the derivation uses the triangle inequality on JS distance, but JS distance is not a metric: the square root of the Jensen-Shannon divergence does not satisfy the triangle inequality in general. This makes the stated linear upper bound unproven. The paper's own Appendix D acknowledges the bound is loose and that empirically the divergence does not scale with K, which further weakens the theoretical contribution. Taken together, these issues do not invalidate the method's potential utility, but they require a conditional verdict with requests for a precise metric definition, error bars, and a corrected/weakened theoretical claim.","tokens_in":17596,"tokens_out":3245,"duration_ms":29817,"concrete_test":"Redefine the efficiency metric in Table 1 in terms of tokens actually read by the model: for each benchmark, report (a) the raw average prompt length in tokens processed during EMLoC's adaptation forward passes (sum of chunk lengths across all chunks, including the system prompt), (b) the average number of stored KV tokens after pruning, and (c) the total inference FLOPs/time for both MLoC and EMLoC under the identical 20-example setting. Also compute the mean and standard deviation of accuracy over three random demonstration-sampling seeds for the five Table 1 benchmarks; if the YouCook2 gap of 6.8 points (108.8 vs 102.0) persists, the claim 'surpasses MLoC with full memory in most benchmarks' must be amended.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central efficiency claim in Section 4.2 is that 'EMLoC dramatically reduces the average context length from 11338 to 2600, a remarkable 77% reduction, without sacrificing performance.' This is the paper's headline quantitative contribution. The concern: the 11338 number is the average length of the full raw prompt (as shown by Table 1's MLoC context lengths, e.g., 16264 for ImageNet100 with 20 examples), whereas the 2600 number is the length of EMLoC's compressed memory, i.e., the number of retained key-value tokens after pruning, not the length of the input that the model actually processes during adaptation. During EMLoC's forward pass, the model still reads the full chunked context (the 16264 tokens) to compute attention weights and hidden states; only the cached KVs are pruned. Thus the 77% figure is a comparison between the uncompressed prompt length and a post-pruning KV-memory length. The two numbers are not the same kind of quantity. The paper does not report the raw input token count that EMLoC actually processes, nor does it report per-benchmark inference FLOPs/time for the 20-example setting in Table 1 (it only reports FLOPs/time for ImageNet100 with 200 examples, where the reduction is 1.76T to 1.35T, only 23%, not 77%). The 'without sacrificing performance' part is also not statistically supported: Table 1 shows differences of 0.1-1.1 points (e.g., 62.6 vs 63.6 on ImageNet100, 41.1 vs 42.2 on MME-RW, and 108.8 vs 102.0 on YouCook2, where EMLoC is actually 6.8 points worse), with no error bars or repeated runs. The YouCook2 result directly contradicts the claim that EMLoC 'surpasses MLoC with full memory in most benchmarks' without noting the degradation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EMLoC, a training-free method for adapting multimodal large language models to downstream tasks by providing in-context demonstration examples. To reduce the computational cost of long contexts, EMLoC partitions demonstrations into chunks, computes a KV cache for each chunk, and then prunes tokens layer-wise using attention-based importance scores under a Jensen-Shannon divergence constraint. The compact memory is then used for inference. The authors report experiments on six vision-language benchmarks with Qwen2-VL, claiming a 77% average context-length reduction at 20 examples without sacrificing accuracy, and they provide a theoretical bound on information loss in terms of the local JS threshold and the number of chunks.","tokens_in":18032,"tokens_out":4460,"duration_ms":50087,"significance":"If the efficiency claims hold, EMLoC is a practical contribution: it offers training-free adaptation with modest memory and latency overhead at adaptation time, and it outperforms static KV-cache pruning baselines such as SnapKV/H2O and PyramidKV on the tested benchmarks. The paper also includes useful ablations on the observation window, retention ratios, chunk sizes, and JS thresholds, and the code is publicly released. However, the central efficiency metric conflates raw prompt length with post-pruning memory size, the theoretical bound rests on unproven or explicitly empirical inequalities, and the claimed performance parity is contradicted by one benchmark and unsupported by error bars. These issues need to be addressed before the main claims can be accepted.","major_comments":[{"comment":"The headline claim that EMLoC reduces the average context length from 11338 to 2600 (a 77% reduction) compares the raw prompt length of MLoC with the post-pruning KV-memory length of EMLoC. These are not the same quantity: during adaptation, EMLoC still performs a forward pass over the full chunked context in Eq. (7) to compute attention weights and hidden states, and only the cached KV pairs are pruned. The paper does not report per-benchmark inference FLOPs or time for the 20-example setting, so the 77% figure does not directly translate into a 77% reduction in end-to-end inference cost. The authors should either report the actual input tokens processed during both phases or clearly define the metric as `context length at generation time after compression`, and provide FLOPs/time for the headline setting.","section":"Section 4.2 and Eq. (7)"},{"comment":"The claim that EMLoC achieves performance on par with or superior to MLoC with full memory is contradicted by the YouCook2 results: MLoC obtains 108.8 with 20 examples, while EMLoC obtains 102.0, a 6.8-point drop. This is not a small difference, and it is not discussed in the text. The statement in Section 4.2 that EMLoC 'surpasses MLoC with full memory in most benchmarks' is technically true only if 'most' is read as five out of six, but the unqualified phrase 'without sacrificing performance' is not supported by the table. The authors should add per-task discussion, explain why YouCook2 degrades, and qualify the performance claim accordingly.","section":"Table 1, YouCook2 row"},{"comment":"The theoretical upper bound in Eq. (16) relies on two inequalities that are not established. Eq. (14) asserts that adding chunks from the full memory monotonically reduces JS divergence; this is a plausible heuristic but is not proved and is not generally true for arbitrary conditional distributions. Eq. (15) is explicitly labeled 'empirically assume', and its direction is not obvious: the left-hand side uses the full demonstration set D while the right-hand side uses only the chunk-specific demonstrations D_k, and there is no reason that conditioning on more data must reduce the JS distance. Since the claimed linear bound is a stated contribution, the authors should either prove these inequalities under explicit assumptions or reframe the analysis as a heuristic motivation and validate Eq. (15) empirically on the evaluated benchmarks.","section":"Section 3.3, Eqs. (14)-(16)"},{"comment":"All reported numbers appear to come from a single run with no error bars or multiple seeds. Given that the 'without sacrificing performance' claim rests on differences of 0.1-1.1 points on several benchmarks, these differences are within the range of normal seed-to-seed variability for MLLM evaluation. The authors should provide standard deviations over at least three runs, or otherwise justify that the comparisons are statistically meaningful. This is particularly important for benchmarks like IllusionVQA where EMLoC and MLoC are exactly tied (40.9 vs. 40.9).","section":"All experimental tables (Tables 1-3, 5-8)"}],"minor_comments":[{"comment":"The text references 'Table 4.2' when comparing with other multimodal ICL methods; this should be 'Table 3'.","section":"Section 4.2"},{"comment":"The phrase 'compresses the context by nearly a quarter' should be 'compresses the context to nearly a quarter of its original length' (or 'by about 78%'), since 3643/16264 is 22.4%.","section":"Section 4.2, Table 2 discussion"},{"comment":"There are minor typos: 'framewrk' in the Figure 2 caption and 'Abalation' in the Table 8 caption should be corrected.","section":"Figure 2 and Table 8"},{"comment":"The MedXpertQA dataset is introduced in the task-disparity analysis without a reference or a description of its construction and evaluation protocol; please add details so the experiment is reproducible.","section":"Appendix C.4"},{"comment":"The axes of Figure 3 are not clearly labeled in the manuscript text; please add explicit axis labels (e.g., JS threshold delta on the x-axis, accuracy/context length on the y-axes) and a legend.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and practically relevant problem, and the empirical comparison against KV-cache pruning baselines is informative. However, the main efficiency claim is currently overstated because the metric mixes prompt length and post-pruning memory size, and the theoretical analysis in Section 3.3 is not a rigorous proof as presented. These are fixable with additional experiments and a more careful framing, so I recommend major revision rather than rejection. Please ask the authors to report end-to-end computational costs for the headline setting, add error bars, and either prove or properly qualify the JS-divergence inequalities."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a real look. The core device is a layer-wise, adaptive pruning loop that uses the Jensen-Shannon divergence between answer-token output distributions before and after pruning as a per-layer stopping rule, on top of chunked compression of multimodal demonstrations. That combination is new relative to the cited static KV-cache methods, and the ablations against SnapKV/H2O and PyramidKV at matched compression ratios are genuinely informative: at 22.4% retention EMLoC holds 63.7 on ImageNet100 while the static methods fall to the high 40s. The JS-gated pruning is a sensible idea and appears to be the main reason for the gap. Code is released, the method is described clearly, and the experiments cover six benchmarks plus a long-video appendix.\n\nThe soft spots are real but not disqualifying. The headline '77% context reduction' compares full prompt length (MLoC) to compressed KV-memory size (EMLoC). That is a fair measure of inference-time context, since after adaptation EMLoC generates from the compact memory only, but it is not total work: the adaptation forward passes still read the full chunked context, and that cost is reported separately only in the appendix (144s vs 28s adaptation time). The main text should state 'inference-time context' and report adaptation overhead in the same table. There are no error bars; several results are within a point or two, and YouCook2 drops from 108.8 to 102.0, which undercuts the 'without sacrificing performance' claim. That dataset needs honest discussion. The theoretical bound in Section 3.3 is heuristic: Eqs. (14) and (15) are asserted, not derived, and Appendix D admits the empirical delta does not grow with K, so the (K−1)√δ+ε bound is not tight. That is fine as an intuition, but it should not be presented as a tight analysis. The circularity concern from our internal read is not real: the JS constraint is an algorithmic input, not a fitted target.\n\nThe central argument holds. JS-constrained adaptive pruning is a practical and promising way to compress multimodal demonstrations, and the inference-time efficiency gains are plausible. This deserves a serious referee. My verdict would be conditional acceptance: the method is useful, but the authors need to fix the metric definition, add repeated runs or error bars, and address the YouCook2 drop directly.\n\nRecommendation: send to peer review, with a request for major revision on the reporting issues.","headline":"A promising adaptive KV-pruning method for multimodal in-context learning, with a misleading headline metric and a few reporting gaps; the core idea deserves a serious referee.","tokens_in":18542,"tokens_out":3583,"would_cite":true,"duration_ms":34227,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EMLoC, a training-free compression method, cuts 20 multimodal demonstrations from 11,338 to 2,600 average tokens—a 77% reduction—while matching or beating full-context accuracy on most benchmarks.","keywords":["training-free adaptation","multi-modal long context learning","KV cache compression","layer-wise adaptive pruning","Jensen-Shannon divergence","in-context learning","vision-language models","chunk-wise compression"],"falsifier":"Take ImageNet100 with 200 demonstrations at the default threshold $\\delta = 0.005$; the claim predicts that the compressed memory keeps accuracy at least as high as the full-context baseline score of 62.6. If accuracy on the full 5,000-image validation set falls below 60.0 while the JS-divergence check still passes, the pruning proxy is not faithful. The same test can be run on any benchmark where EMLoC's compressed memory is compared against full-context MLoC.","tokens_in":17404,"feed_emoji":"✂️","tokens_out":13861,"duration_ms":120972,"temperature":0.7,"pith_summary":"The paper tries to establish that a multimodal large language model can be adapted to a new task without any training by feeding it demonstration examples and then compressing those examples into a compact task-specific memory. The proposed method, EMLoC, splits the long demonstration context into chunks and prunes each chunk's key-value cache layer by layer, guided by answer-token attention and a Jensen-Shannon divergence check. On six vision-language benchmarks, the compression reduces the average context length from 11,338 to 2,600 tokens with 20 examples—a 77% reduction—while matching or beating the full-context baseline on most benchmarks. A reader should care because it offers a training-free, plug-and-play route to few-shot adaptation at a fraction of the inference cost.","feed_headline":"Context pruning cuts multimodal demos 77% with no accuracy loss","feed_subtitle":"A training-free method turns long demonstrations into a small memory, matching or beating full-context accuracy on six benchmarks","key_machinery":"The central object is the compressed task-specific memory $\\mathcal{M}$, the surviving KV cache produced from the demonstration chunks. Chunk-wise compression splits the long context into $K$ chunks (default 1.6k tokens) so each fits on one GPU; within each chunk, layer-wise adaptive pruning retains, per layer $l$, the top $r \\times S$ tokens by accumulated answer-token attention $\\beta^l_j$, trying retention ratios from an ascending set until the Jensen-Shannon divergence—a measure of how much two probability distributions differ—between original probabilities $p^{\\mathrm{ori}}$ and iteratively pruned probabilities $p^{\\mathrm{iter}}$ stays below $\\delta$. The supporting theoretical result is a JS-distance triangle inequality giving a global upper bound that scales linearly with chunk count and with $\\sqrt{\\delta}$, which is what lets the paper treat the local pruning threshold as the single hyperparameter controlling the accuracy/compression trade-off.","core_discovery":"On its own terms, the paper's central claim is that long multimodal demonstration contexts are highly redundant and that the redundancy can be removed greedily without retraining. EMLoC scores every token in a chunk's KV cache by the attention it receives from answer tokens, then prunes the least-important tokens in each layer from top to bottom, trying successively larger retention ratios until the Jensen-Shannon divergence between the original and pruned answer-token output distributions falls below a threshold $\\delta$. The final compressed memory preserves task accuracy, and on most benchmarks it matches or slightly exceeds full-context performance because pruning removes irrelevant background noise. The paper derives a linear upper bound on the global information loss, $D_{\\mathrm{JS}}(P^{\\mathcal{D}}_{\\mathcal{M}}, P^{\\mathcal{D}}_{\\mathcal{M}^K}) \\le (K-1)\\sqrt{\\delta} + \\epsilon$, to justify using the local per-layer constraint as a global control knob.","pith_inferences":["If the JS-divergence proxy generalizes beyond the six tested benchmarks, the same compression could be applied to the demonstrations retrieved by an online retrieval method, turning a five-hour, 43G-memory retrieval-augmented inference procedure into one that takes minutes and 18G.","The observed stability of the global JS divergence as the chunk count $K$ grows suggests inter-chunk dependencies are weak in practice; a testable extension is to push $K$ much higher and check whether the effective bound $\\Delta \\le \\gamma\\delta$ with $\\gamma \\le 2$ continues to hold, which would let memory usage scale down almost arbitrarily with chunk size.","Since image tokens make up the majority of pruned tokens, a natural follow-up is to assign modality-specific thresholds or retention ratios; one could test whether treating visual and text tokens separately improves the compression/accuracy trade-off beyond the single-threshold version."],"forward_implications":["With 20 demonstration examples, EMLoC uses 2,600 tokens on average instead of 11,338, a 77% reduction, and on most benchmarks it matches or exceeds full-context accuracy.","On ImageNet100 with 200 examples, EMLoC reaches 63.7% accuracy with 3,643 tokens, beating the full-memory MLoC score of 62.6% at 16,264 tokens.","Inference cost falls with the context: on ImageNet100, LLM FLOPs drop from 1.76T to 1.35T and total inference time from 1,866s to 1,107s.","Adaptation remains training-free and fast: 144s for EMLoC versus 234s for LoRA and 820s for full fine-tuning, with average accuracy 54.9 versus 54.7 for LoRA across ImageNet100, MME-RW, and OK-VQA.","The layer-wise pruning profiles show that layer importance is not a fixed pyramid; layers 4, 8, and 14 of Qwen2-VL retain markedly more tokens than their neighbors, so fixed-ratio pruning loses accuracy where adaptive pruning does not."],"supporting_citations":[{"why":"Qwen2-VL is the base MLLM used for all main experiments; its 32k multimodal context window and resolution-flexible vision encoder make the long-context setup possible.","marker":"Wang et al., 2024b"},{"why":"StreamingLLM's finding that attention concentrates on a few tokens motivates using accumulated attention as the token-importance signal for pruning.","marker":"Xiao et al., 2024"},{"why":"SnapKV is the KV-cache pruning baseline that EMLoC is compared against and whose fixed-retention approach EMLoC extends to per-layer adaptivity.","marker":"Li et al., 2024"},{"why":"PyramidKV supplies the pyramid-shaped pruning baseline and the claim that earlier layers matter more, which EMLoC's layer-wise adaptive profiles challenge.","marker":"Cai et al., 2024"},{"why":"Flamingo's RICES is the retrieval-augmented many-shot multimodal ICL baseline compared in Table 3.","marker":"Alayrac et al., 2022"},{"why":"MTV is the task-vector many-shot ICL baseline compared in Table 3.","marker":"Huang et al., 2024a"},{"why":"LoRA is the parameter-efficient fine-tuning baseline whose adaptation time and accuracy EMLoC compares against.","marker":"Hu et al., 2022"},{"why":"Its study of many-shot in-context learning in multimodal foundation models motivates the target of enabling long-context adaptation for open-source MLLMs.","marker":"Jiang et al., 2024"},{"why":"ImageNet-1k supplies the ImageNet100 benchmark on which the 200-example compression and accuracy results are measured.","marker":"Deng et al., 2009"}],"fun_headline_variants":["Training-free adaptation via chunk compression and layer pruning","No fine-tuning needed: prune long demos for multimodal tasks","Layer-wise token pruning compresses multimodal demos, no fine-tuning","Adaptive pruning cuts demo context 77% and keeps accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"EMLoC rests on the premise that answer tokens are a faithful observation window—tokens they attend to are the ones the task needs—and that keeping the Jensen-Shannon divergence of answer-token probabilities below $\\delta$ always preserves task accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Training-free adaptation via chunk compression and layer pruning","No fine-tuning needed: prune long demos for multimodal tasks","Layer-wise token pruning compresses multimodal demos, no fine-tuning","Adaptive pruning cuts demo context 77% and keeps accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001241,"raw_usage":{"total_tokens":5097,"prompt_tokens":953,"completion_tokens":4144,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":4075}},"tokens_in":569,"tokens_out":4144,"duration_ms":29986,"temperature":1.0,"reasoning_tokens":4075,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:05:30.631294+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take ImageNet100 with 200 demonstrations at the default threshold $\\delta = 0.005$; the claim predicts that the compressed memory keeps accuracy at least as high as the full-context baseline score of 62.6. If accuracy on the full 5,000-image validation set falls below 60.0 while the JS-divergence check still passes, the pruning proxy is not faithful. The same test can be run on any benchmark where EMLoC's compressed memory is compared against full-context MLoC.","supporting_citations":[{"cited_title":"Flamingo : A visual language model for few-shot learning","cited_arxiv_id":null,"evidence_quote":"Flamingo's RICES is the retrieval-augmented many-shot multimodal ICL baseline compared in Table 3."},{"cited_title":"A., Wang, J","cited_arxiv_id":null,"evidence_quote":"Its study of many-shot in-context learning in multimodal foundation models motivates the target of enabling long-context adaptation for open-source MLLMs."}],"review_version":1}