{"id":"ffc25661-1d2c-447c-86ba-6c682786d3f5","arxiv_id":"2504.14200","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"KeCO updates the visual feature keys of a small coreset with all leftover support images, and its diversity-based update outperforms retrieval from the five-times-larger full support set for LVLM in-context image classification.","lead":"KeCO takes a small, randomly chosen set of example images and folds information from all the unused images into averaged feature keys, making the small set a better in-context teacher for vision-language models. In tests it beat storing and searching the full support set, while using far less memory.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"KeCO's only channel for using untapped data is the updated key, which reranks the same original coreset images the LVLM actually sees; the paper reports no retrieval-precision or key-collapse diagnostics, so the claimed advantage could be a centroid-ranking artifact rather than the Eq.","rationale":"The reader's weakest assumption points to encoder-LVLM alignment and feature interpolation. I agree that this is the crux, but I would sharpen it: the updated key is only a retrieval index; the demonstration image content is never modified. Therefore the paper needs retrieval-side measurements to distinguish the proposed diversity mechanism from a simpler class-centroid ranking effect. This is a genuine gap in support, not a demonstrated error. The headline accuracy result is consistent across the reported cells and does not require rejection, but the mechanism claim and the generalization claim are under-supported without the proposed diagnostics. The reader's CONDITIONAL verdict remains appropriate, so no change is recommended.","tokens_in":20220,"tokens_out":15553,"duration_ms":149975,"concrete_test":"On CUB-200 and Stanford Dogs, log the top-2/top-4 demonstrations selected by FS-IC, KeCO-RS, KeCO-DS, and FS-IS for identical test images. Compute (i) retrieval precision@k and class purity against ground-truth labels, (ii) the average pairwise cosine similarity of the five updated keys per class, and (iii) the accuracy of a k-NN or class-mean classifier built directly on the updated keys. If KeCO's retrieval precision is not higher than FS-IS's, or if within-class key similarity exceeds about 0.95 while replacing KeCO's top-k selection with random selection inside the class predicted by the updated-key nearest centroid reproduces the LVLM accuracy, the claimed diversity-based mechanism is not supported and the FS-IS advantage should be attributed to class-centroid retrieval.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that KeCO outperforms FS-IS with a five-times-smaller support set. For this to be a property of KeCO's update mechanism rather than of the particular CLIP/SigLIP similarity geometry, the updated keys must retrieve demonstrations the frozen LVLM can actually use. Section 4.4's explanation (Figure 4) says averaging dissimilar keys 'preserves category-relevant information' and 'blurs background,' but the LVLM never sees the averaged key; it sees the original image of the selected coreset sample (Section 3.7). Thus all untapped information enters only through the ranking of a fixed class-balanced set of m/j=5 original images per class. The paper gives no retrieval-side evidence: no precision@k, no class-purity of retrieved demonstrations, no measurement of how much the updated keys change the ranking, and no report of within-class key collapse. If the five updated keys per class converge toward the class mean, top-k selection degenerates to near-random choice within the predicted class, and the gain over FS-IS would be explained by a centroid-style class selector rather than by the diversity-based selection in Eq. 7. In that case the Section 4.4 mechanism is not load-bearing, and the result may not transfer to other LVLMs or encoders where such a centroid index is less aligned. This is closely related to the reader's alignment concern, but the missing retrieval diagnostics are the direct, testable gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KeCO (Key-based Coreset Optimization), a method for multimodal in-context learning (ICL) in image classification. Starting from a randomly selected, class-balanced coreset from a support set, KeCO uses the remaining untapped support samples to update a stored visual feature (key) for each coreset image via linear interpolation, with the target coreset sample chosen by random (RS), similarity-based (SS), or diversity-based (DS) selection. At inference, demonstrations are retrieved by ranking the coreset images according to the updated keys, and the original images of the top-k coreset samples are fed to a frozen LVLM. Experiments on CUB-200, Stanford Dogs, and ImageNet-100 with OpenFlamingo-3B, IDEFICS-8B, and Qwen2-VL report that KeCO-RS and KeCO-DS outperform both a fixed coreset baseline (FS-IC) and a full-support-set similarity retrieval baseline (FS-IS), with KeCO-DS best and KeCO-SS worst. A simulated online variant is also evaluated. The central empirical claim is that updating keys with untapped data yields a compact coreset that beats five-times-larger support set retrieval.","tokens_in":20427,"tokens_out":4329,"duration_ms":40416,"significance":"If the empirical claims hold, KeCO is a simple, memory-efficient way to improve LVLM ICL for image classification, with a plausible practical advantage in streaming/online settings. The method is easy to implement, the code is provided, and the reported gains over FS-IS are large in several configurations (e.g., OF-3B 2-shot CUB-200: 74.20 vs 61.04). The paper also makes a useful conceptual point that discarded support-set samples need not be wasted. However, the strength of the contribution is currently tempered by the absence of any variance or statistical significance information across runs, and by the lack of direct evidence for the proposed retrieval-quality mechanism. The comparison ordering DS > RS > FS-IS is consistent across datasets and models, which lends some credibility, but single-run numbers and the absence of retrieval diagnostics leave the mechanism claim under-supported.","major_comments":[{"comment":"All reported accuracies are single-run values with no error bars, no multiple random coreset initializations, and no significance tests. The central claim that KeCO-DS outperforms FS-IS relies on differences that, while large for OF-3B, are small for IDE-8B (e.g., CUB-200 2-shot: 87.38 vs 85.62; ImageNet-100 2-shot: 91.68 vs 90.48). Without repeated trials and a measure of variance, it is impossible to assess whether the apparent improvements are robust or partly due to the particular random initialization of the coreset. Please report mean and standard deviation over at least 3-5 seeds and, where feasible, a paired significance test.","section":"Section 4.3, Table 1"},{"comment":"The paper claims that the improved performance is caused by updated keys retrieving more category-relevant demonstrations (Figure 4), but no retrieval-side evidence is provided. There is no measurement of precision@k, no class purity of the retrieved demonstrations, no analysis of how much the updated keys change the ranking, and no diagnostic for key collapse within a class. Since the LVLM only ever sees the original images of the coreset samples (Section 3.7), all untapped information enters through the reranking. If the five updated keys per class converge toward a class mean, the top-k selection may degenerate to a centroid-style class selector rather than the diversity-based mechanism claimed in Eq. (7). Please add quantitative retrieval diagnostics, such as the rank correlation between updated and original keys, the fraction of retrieved demonstrations matching the test label, and a dispersion measure of keys within each class before and after updating.","section":"Section 4.4, Figures 3-5"},{"comment":"The abstract claims an 'average improvement of more than 20%.' This figure is not precisely defined: it appears to be a relative improvement over FS-IC for OF-3B on particular configurations, but the exact averaging set and whether it is relative or absolute is not stated. As written, the claim is ambiguous and could mislead readers. Please specify the exact baseline, the configuration subset, and whether the number is a relative or absolute improvement, or replace it with a table summary.","section":"Abstract and Section 4.4"},{"comment":"The hyperparameters alpha, epoch count e, and batch size b are tuned on CUB-200 (Tables 4-5) and the best values are then used for all datasets and models, but there is no validation split and no discussion of how selection bias might inflate the reported numbers. Because the same test sets are used for both hyperparameter selection and final evaluation, the magnitude of the reported gains may be optimistic. Please either perform hyperparameter selection on a held-out split or report sensitivity curves for all datasets, and state clearly whether the reported numbers used the best hyperparameters on each dataset or a single fixed setting.","section":"Section 4.3 and Appendix A.3"}],"minor_comments":[{"comment":"Typo: 'everages' should be 'leverages'.","section":"Conclusion"},{"comment":"Typo: 'optimzie' should be 'optimize'; also 'perforamce' in Table 4 header should be 'performance'.","section":"Section 4.4"},{"comment":"The t-SNE visualization is qualitative. Adding a quantitative separability measure, such as silhouette score or mean intra-class to inter-class distance ratio, would strengthen the claim that keys become more clustered.","section":"Figure 3"},{"comment":"The column header 'Online Δ' is unclear; specify whether the numbers are absolute accuracies of KeCO-DS in the online setting and what the delta refers to.","section":"Table 2"},{"comment":"References [16] and [17] appear to be the same paper ('Obelics') with different years; please merge or disambiguate.","section":"References"},{"comment":"The class-balance constraint '|C_c| = m/j' assumes m is divisible by j; when this is not the case, the text should specify how the remainder is assigned.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the code availability is a plus. The main issue is that the empirical claim, while directionally credible, is supported only by single-run numbers, and the proposed mechanism is not directly tested. The authors should be encouraged to add variance information and retrieval diagnostics. I do not see a fundamental flaw in the approach; the weaknesses appear fixable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nShort take: KeCO does something genuinely new—it updates stored visual “keys” of a small coreset using features from the untapped support set, then retrieves demonstrations by those updated keys. The strongest finding is that diversity-based target selection (least similar, Eq. 7) clearly beats similarity-based selection (Eq. 6), and KeCO-DS consistently beats both FS-IC and FS-IS across three datasets and two LVLMs. That is a real result, and the online scenario extension is a nice practical touch.\n\nWhat is good: the method is straightforward, the code is released, and the experiments are broad enough to make the central claim credible. The headline “more than 20% improvement” is best-case cell selection, but the underlying pattern is consistent.\n\nSoft spots, in order of importance:\n\n1. The mechanism story in Sec. 4.4 is not supported by the evidence reported. The LVLM never sees the averaged keys; it sees the original images of the selected coreset samples. All the untapped information enters only through reranking a fixed set of m/j original images per class. The paper gives no precision@k, no class-purity of retrieved demonstrations, no analysis of how much the keys actually change the ranking, and no report of within-class key collapse. If the five keys per class converge to the class mean, top-k retrieval becomes near-random selection within a predicted class, and the gain over FS-IS might be explained by a centroid-style class selector, not by the diversity mechanism. That wouldn't kill the empirical result, but it would change the story. The stress-test note lands: this is the direct testable gap.\n\n2. Hyperparameter tuning is not clean. alpha=0.2 is selected on CUB-200, and CUB-200 appears in the headline Table 1. The ablation in App. A.3 shows OF-3B online CUB 2-shot at alpha=0.2 is 76.96, while Table 1 online KeCO-DS for the same setting is 70.61. That is a large, unexplained discrepancy, and it suggests the reported numbers are not stable across repeated runs. No variance or seeds are reported anywhere.\n\n3. Baseline coverage is thin. No existing coreset method (e.g., the InfoScore method they cite) is actually run as a baseline; they only compare against random-coreset and full-support retrieval.\n\n4. Minor: some claims in the abstract and intro are best-cell summaries, and there is an odd 2-shot-to-4-shot drop for OF-3B on CUB-200 in the FS-IC and KeCO-SS columns of Table 1.\n\nNone of this is fatal. The central direction is credible, the method is simple and reproducible, and the diversity-selection finding is worth reporting. With error bars, a rerun of the alpha ablation on a held-out dataset, and a retrieval-side diagnostic, this could be a solid contribution.\n\nWho this is for: anyone working on efficient ICL for LVLMs or coreset-based demonstration selection. It deserves a serious referee, but I would ask for the diagnostics before accepting.\n\nBest,\n[You]","headline":"Solid engineering paper with a credible central result, but the mechanism story is under-supported and there are reporting inconsistencies that need fixing before the claims are fully trustworthy.","tokens_in":21111,"tokens_out":2771,"would_cite":false,"duration_ms":23253,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Discarded support images, folded into a small coreset, beat full-set in-context retrieval.","keywords":["in-context learning","large vision-language models","coreset optimization","image classification","demonstration selection","diversity-based selection","online learning","visual feature keys"],"falsifier":"A concrete check: swap the backgrounds of same-class images and see whether diversity-updated keys still retrieve the correct class; if background similarity, not class, drives the updated key, the mechanism is refuted. Alternatively, rerun KeCO-DS with a vision encoder that is not aligned with the model's internal representation and check whether the 10–20% gains persist; vanishing gains would show the alignment premise, not the update rule, is doing the work.","tokens_in":19901,"feed_emoji":"🖼️","tokens_out":7879,"duration_ms":66761,"temperature":0.7,"pith_summary":"The paper proposes KeCO, a method for building a small demonstration set—a coreset—for in-context image classification with large vision-language models. Instead of exhaustively retrieving examples from the whole support set, KeCO randomly initializes a compact coreset and then updates each core sample's visual feature, called its key, by blending it with unused support images of the same class. The paper's central claim is that this feature-level update lets a coreset one-fifth the size of the support set outperform retrieval from the full support set, lifting OpenFlamingo-3B accuracy by 20% or more relative to the fixed coreset and IDEFICS-8B accuracy by about 10%. The update works best when each unused image is matched to the most dissimilar same-class core sample, because repeated averaging preserves shared class-typical features while blurring background, pose, and lighting. The same procedure also works when data arrives as a stream, which is the practically relevant setting for memory-constrained deployment.","feed_headline":"Untapped images beat five-times-larger support sets","feed_subtitle":"KeCO distills discarded support images into updated keys, lifting CUB-200 2-shot accuracy from 61% to 74%.","key_machinery":"The central object is the key $k_t=\\phi(I_t)$, the visual feature of a coreset image extracted by the model's vision encoder (CLIP/ViT-L-14 for OpenFlamingo, SigLIP for IDEFICS). The update rule is $k'_{t_j}=k_{t_j}-\\alpha\\,\\frac{1}{|S'_{i,j}|}\\sum_{s\\in S'_{i,j}}(k_{t_j}-\\phi(I_s))$, which moves each core key partway toward the mean feature of the untapped samples assigned to it. The argument's load-bearing pairing is this update with diversity-based target selection: choosing the least similar same-class core sample means averaging across dissimilar images, so the class-consistent signal survives while per-image background, pose, and illumination differences cancel. The paper's mechanism claim is that this makes keys more clustered by category, and that clustering is what lets retrieval from the updated coreset give the frozen LVLM cleaner demonstrations.","core_discovery":"On the paper's own terms, the discovery is that discarded support samples are not waste: their features can be folded into a much smaller coreset, and the choice of which core sample each discarded sample updates determines whether the fold helps or hurts. With diversity-based selection, each untapped image updates the least similar same-class core sample, and the updated keys become more tightly clustered by class, so top-k retrieval from the updated coreset supplies the frozen LVLM with more informative demonstrations. The strongest quantitative claim is that this updated coreset of about 1,000–1,200 samples beats the fixed full support set of 5,000–6,000 samples: on CUB-200, OpenFlamingo-3B rises from 61.04% to 74.20% in the 2-shot setting and from 58.61% to 76.99% in the 4-shot setting. Similarity-based selection, by contrast, tends to retain misleading background information and can underperform even the unrefined coreset. The paper also claims the same update transfers to a simulated online setting where each sample is seen once, including a gain of roughly 5 points for Qwen2-VL on CUB-200.","pith_inferences":["Editorial inference: KeCO is effectively a feature-space data-distillation step, so the same update rule could be applied to other retrieval-based multimodal tasks, such as captioning or visual question answering, whenever the retriever's encoder is also the model's encoder; the diversity rule would need to be revalidated per task.","A testable extension the paper does not run: measure how the gain saturates as the untapped set grows; the paper's ratio study (1:2 to 1:6) shows continuing improvement, so finding the saturation point would tell practitioners how much unlabeled support data is worth collecting.","One risk not resolved by the paper's experiments: updated keys are interpolated feature vectors that may not correspond to any real image, so if the vision encoder and the language model are weakly aligned, the benefit could degrade; the paper's own comparison with Infoscore initialization hints that such misalignment exists."],"forward_implications":["A coreset of size 1,000–1,200 can beat full-support retrieval from 5,000–6,000 samples on the tested benchmarks, so the memory and search cost of ICL demonstration selection can be cut by roughly a factor of five without sacrificing accuracy.","Diversity-based selection is the rule that makes the update useful; random selection helps, but similarity-based selection can hurt, so the mechanism behind KeCO is the pairing of averaging with dissimilar partners.","The framework carries over to online and streaming data, where each sample is used once; this turns coreset construction into a continual update process rather than a one-time offline selection.","Fine-grained datasets benefit more than coarse-grained ones, consistent with the claim that updated keys supply class-discriminative visual knowledge the LVLM lacks.","The initialization strategy matters less than the update: even a random class-balanced coreset, after KeCO updates, outperforms coresets initialized by k-center or Infoscore in most compared settings."],"supporting_citations":[{"why":"Defines the Flamingo LVLM architecture whose few-shot capabilities KeCO is built to improve.","marker":"[1]"},{"why":"Provides OpenFlamingo-3B, the primary open LVLM used in the main experiments.","marker":"[2]"},{"why":"Provides IDEFICS-8B, the second LVLM used in the main experiments.","marker":"[16]"},{"why":"Supplies the CLIP/ViT-L-14 vision encoder whose features serve as coreset keys for OpenFlamingo.","marker":"[36]"},{"why":"Supplies the SigLIP vision encoder whose features serve as coreset keys for IDEFICS.","marker":"[49]"},{"why":"Supplies Qwen2-VL, the high-performance LVLM used in the simulated online scenario.","marker":"[41]"},{"why":"Supplies the Infoscore coreset-selection baseline from NLP that KeCO adapts and improves upon with random initialization.","marker":"[26]"},{"why":"Supplies the CUB-200 fine-grained bird dataset where the largest KeCO gains are reported.","marker":"[42]"},{"why":"Supplies the Stanford Dogs fine-grained dataset used in the main comparisons.","marker":"[15]"},{"why":"Supplies ImageNet-100, the coarse-grained dataset used to compare fine-grained versus coarse-grained gains.","marker":"[38]"}],"fun_headline_variants":["Discard smartly: KeCO folds unused images into a tiny coreset","KeCO: Updating coreset keys with discarded images lifts 2-shot accuracy 13 points","From 61% to 74%: KeCO makes small coresets beat full support sets","Coreset keys updated by discarded samples outperform 5x larger sets","KeCO's diversity-based key updates turn waste into accuracy gains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a key updated by averaging a core image's feature with a dissimilar same-class image retains the class signal and erases the differences, and that this feature space is aligned well enough with the frozen LVLM's internal representation that the updated keys order demonstrations the model actually uses.","fun_headline_variants_meta":{"raw":{"variants":["Discard smartly: KeCO folds unused images into a tiny coreset","KeCO: Updating coreset keys with discarded images lifts 2-shot accuracy 13 points","From 61% to 74%: KeCO makes small coresets beat full support sets","Coreset keys updated by discarded samples outperform 5x larger sets","KeCO's diversity-based key updates turn waste into accuracy gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000796,"raw_usage":{"total_tokens":3553,"prompt_tokens":1045,"completion_tokens":2508,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":2403}},"tokens_in":661,"tokens_out":2508,"duration_ms":16911,"temperature":1.0,"reasoning_tokens":2403,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:55:25.033310+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: swap the backgrounds of same-class images and see whether diversity-updated keys still retrieve the correct class; if background similarity, not class, drives the updated key, the mechanism is refuted. Alternatively, rerun KeCO-DS with a vision encoder that is not aligned with the model's internal representation and check whether the 10–20% gains persist; vanishing gains would show the alignment premise, not the update rule, is doing the work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides IDEFICS-8B, the second LVLM used in the main experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Stanford Dogs fine-grained dataset used in the main comparisons."}],"review_version":1}