{"id":"00bd1ecd-0d76-44db-82cb-f61e30a835e7","arxiv_id":"2504.21263","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Condenser fuses multiple candidate prompts into one condensed prompt and improves visual in-context learning accuracy on three benchmark tasks.","lead":"Visual in-context learning lets a vision model solve new tasks by looking at example images. This paper introduces Condenser, a small module that fuses several example prompts into one, and reports gains on segmentation, detection, and colorization while keeping inference fast.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Patch-wise cross-attention (Eqs. 6–9) ties each query patch to the same spatial location in every candidate prompt, so the central claim that Condenser integrates context from arbitrary prompts is untested for translated or misaligned prompt content.","rationale":"The reader's weakest_assumption is the same one I would flag: the patch-wise cross-attention in Eqs. (6)–(9) assumes that for every query patch the only useful prompt context is at the identical spatial position. This is a genuine architectural constraint, not just an evaluation gap, and it directly bears on the advertised benefit of condensing multiple prompts. I agree that this is the most load-bearing concern because it targets the mechanism the paper introduces, rather than secondary issues such as missing error bars or the absence of a validation-split description. Those issues are real but they affect confidence intervals and reproducibility, not the validity of the central construction. The paper has independent support: open-source code, standard benchmark comparisons, ablations, and monotonic improvement with K all substantiate that Condenser works on the tested distributions. The proposed check would determine whether the spatial-locking assumption is a benign inductive bias or a hard ceiling on the claimed generalization. Until that check is run, the conditional verdict remains appropriate; I would not move to reject or accept on the current evidence.","tokens_in":28170,"tokens_out":4268,"duration_ms":52192,"concrete_test":"Using the released code, build a synthetic segmentation evaluation on Pascal-5i: for each query, keep the top-K retrieved prompts but apply a random spatial translation (e.g., shift the object and its mask by 1/8 of the image) to every prompt image and label before feeding them to Condenser. Compare Condenser K=16 against Condenser K=1 and against the full-cross-attention variant (3). If K=16 no longer improves over K=1, or if full cross-attention becomes better, the patch-wise spatial restriction is the cause and the multi-prompt claim is contingent on layout alignment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the spatial-locking assumption in the Condenser design, explicit in Sec. 3.3 and Eqs. (6)–(9). For a query patch at (h,w), cross-attention over prompt images and labels is restricted to patches at the same (h,w) in every candidate prompt. This means Condenser can never use task-relevant cues located elsewhere in a prompt, such as an object that is translated, at a different scale, or accompanied by an annotation offset. The paper's own ablation variant (3), replacing patch-wise with full cross-attention, is worse on the standard benchmarks, so the design has in-distribution empirical support; however, that ablation does not test the regime where the assumption is most brittle. Since candidate prompts are retrieved by whole-image similarity, they may often share layout with the query, but the paper's claim is broader: that candidate prompts collaborate to contribute 'relevant fine-grained context' regardless of where that context sits. Under the stated equations, any useful cue that is spatially shifted is simply discarded. This does not make the method internally inconsistent, but it caps the generalization of the central multi-prompt-integration claim. The scaling with K in Fig. 5 and the K=1 gains show that Condenser learns a useful feature transformation, but the specifically multi-prompt benefit advertised in the abstract is precisely what the spatial-locking assumption puts at risk.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Condenser, a lightweight external module for visual in-context learning (VICL). Instead of selecting one prompt, Condenser fuses K retrieved image-label prompt pairs into a single condensed prompt at patch-token level. The module applies shared self-attention to prompt images and labels and then patch-wise cross-attention, with the query image as the attention query (Eqs. (1)-(9)). It is trained end-to-end with a frozen MAE-VQGAN backbone using a token-prediction loss (Eq. (13)) and a pre-alignment cosine loss (Eq. (14)). Experiments compare Condenser with prompt-selection, voting, and PEFT baselines on Pascal-5i segmentation, Pascal VOC detection, and ImageNet colorization, plus cross-dataset and efficiency studies. The paper claims consistent gains, scalability with K, and lower inference cost than output ensembling.","tokens_in":28418,"tokens_out":5706,"duration_ms":55561,"significance":"If the headline numbers hold, collaborative prompt condensation is a plausible new direction for VICL and a useful alternative to output ensembling. The method is clearly specified, the ablation studies cover the main design choices (PCA, losses, output fusion, PEFT), and the code is open-sourced. I do not see circularity: the query label is used as a training target, which is standard supervision. However, the strength of the empirical evidence is currently below what the paper's broad claims require: the spatial-locking assumption is untested, the key comparisons lack error bars and partly reuse other papers' numbers, and the cross-dataset multi-prompt gain is negligible. The contribution is significant conditional on those points being addressed.","major_comments":[{"comment":"Patch-wise cross-attention restricts the output for query patch (h,w) to attend only to prompt patches at the same (h,w), for both image and label streams. This means any task-relevant cue that appears at a shifted position in a candidate prompt (e.g., an object translated, a scale change, or an annotation offset) is discarded by construction. The ablation in Table 4, Variant (3), shows that full cross-attention is worse on the standard benchmarks, which supports the local-consistency rationale in-distribution, but it does not test the regime where the spatial-locking assumption is most brittle. Since the paper's central claim is that Condenser compresses relevant fine-grained context across multiple prompts from arbitrary candidates, please add an experiment with translated or scaled prompt content (or retrieve prompts with deliberately misaligned layout) and report whether the advantage over K=1 and over full cross-attention degrades; otherwise, the claim should be explicitly restricted to spatially aligned prompts.","section":"Sec. 3.3, Eqs. (6)-(9)"},{"comment":"The main empirical claims rest on a comparison in which no error bars or seeds are reported, and several baseline numbers appear to be copied from previous papers without shared reimplementation. The detection result illustrates the problem: Condenser with K=1 reaches 43.22 mIoU versus InMeMo's 43.21, i.e. a difference of 0.01, and the colorization gain is 0.02 MSE; such differences are within run-to-run noise. Please report mean and standard deviation over at least three random seeds for Condenser, state explicitly which baseline numbers were re-run under the same protocol and which were taken from the literature, and reconsider the wording 'consistently outperforms' for the cases where the margin is at noise level.","section":"Table 1, Sec. 4.1.3"},{"comment":"The cross-dataset experiment is presented as showing that condensing multiple prompts gives 'remarkable advantages' over single-prompting, but the numbers do not support that wording: K=16 gives 40.52 mIoU versus 40.37 for K=1, and on Fold-3 K=16 is worse (35.84 vs 36.33). This is a small single-run difference, and it is exactly the setting where the claimed generalization benefit of collaboration should appear. Please provide per-seed breakdowns and either temper the claim or show that the multi-prompt gain is consistent across folds and seeds.","section":"Sec. 4.2.3, Table 2"}],"minor_comments":[{"comment":"The word 'goad' should be 'goal' in the sentence 'To achieve this goad.'","section":"Sec. 3.1"},{"comment":"The term 'impainting' in the Figure 1 caption should be 'inpainting', and the label 'V oting' in Table 1 should be 'Voting'.","section":"Figure 1 and Table 1"},{"comment":"References [4] and [5] are the same paper, and references [17] and [32] are also duplicates; these should be merged to avoid citation inflation.","section":"References"},{"comment":"The efficiency conclusion is driven by inference time, but the GPU cost for Condenser K=16 (1021.86 MB) is more than twice that of Prompt-SelF (446.71 MB) and InMeMo (497.13 MB); the text should qualify the efficiency claim as time-efficient but memory-heavier.","section":"Table 3, Sec. 4.2.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful, honest paper for the visual in-context learning crowd. The core idea – condense multiple candidate prompts into one integrated prompt rather than competing for the single best – is a nice reframing, and Condenser itself is a lightweight, well-specified plugin. The gains over single-prompt selection and output ensembling are consistent across segmentation, detection, and colorization, and the ablations give real insight: the token-prediction loss is doing the heavy lifting, pre-alignment helps a bit, and full cross-attention actually hurts, which supports their patch-wise design choice. Code is open, so the work feels reproducible.\n\nThe soft spots are real but they are the kind that a serious revision can fix, not deal-breakers. First, there are no error bars or multiple seeds. The improvements are a few mIoU points, so I would not be comfortable calling the SOTA claim robust without variance estimates. Some baseline numbers appear to be taken from previous papers rather than re-run under identical conditions – common in this area, but it weakens the comparison. Second, there is no direct multi-prompt downsampling baseline, which is the most obvious alternative for putting more prompts on the canvas at full resolution. The mean-pooling ablation covers some of that ground, but not the resolution trade-off.\n\nThe stress-test note about Eqs. (6)–(9) is correct and worth stating plainly: Condenser's cross-attention is spatially locked, so a useful cue at a different location in a prompt is simply ignored. That is a genuine limitation on the generality of the “collaboration” claim, and the title/abstract oversell it a bit. That said, the paper explicitly frames this as a design choice, and the full-attention ablation is empirically worse on the standard benchmarks, so the inductive bias is justified for those tasks. Still, I would want to see a robustness experiment with shifted or misaligned prompts before believing the broader claim about arbitrary candidate prompts.\n\nWho is this for? Researchers working in VICL, prompt tuning, and efficient in-context adaptation. The paper deserves a serious referee: the method is clean, the writing is clear, the experiments are mostly well-designed, and the open-sourced code lowers the barrier to verification. My recommendation is to send it to review, and to ask the authors for error bars, a downsampling comparison, and a direct test of spatial alignment. If those come back okay, this is a solid accept for a workshop or a borderline accept at a main conference.","headline":"A solid, reproducible VICL paper that reframes prompt selection as prompt condensation and backs it with clean ablations, but whose patch-wise attention design quietly assumes spatial alignment between query and prompts.","tokens_in":690,"tokens_out":859,"would_cite":true,"duration_ms":39137,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Condensing multiple candidate prompts into one fine-grained prompt, rather than selecting a single 'ideal' prompt, improves visual in-context learning on segmentation, detection, and colorization while remaining faster than output…","keywords":["Visual in-context learning","Prompt condensation","Prompt selection","Patch-wise cross-attention","MAE-VQGAN","In-context learning","Segmentation","Multi-prompt integration"],"falsifier":"Take a trained Condenser and evaluate it on Pascal-5i after applying a random spatial shift of 16-32 pixels to the prompt images and labels at test time. A human can still map the shifted prompt to the query, but Condenser cannot see cross-position matches; a large accuracy drop relative to unshifted prompts would show the same-position assumption is load-bearing.","tokens_in":27969,"feed_emoji":"🧩","tokens_out":7226,"duration_ms":69666,"temperature":0.7,"pith_summary":"The paper proposes that visual in-context learning should stop treating candidate prompts as competitors and instead condense several good-enough prompts into one. Its Condenser plugin compresses the fine-grained context of multiple prompts at the input level, then feeds the single condensed prompt to a frozen MAE-VQGAN inpainting backbone that predicts the query label. On Pascal-5i foreground segmentation, 16 condensed prompts reach 46.63 mIoU against 43.14 for the strongest baseline, while costing 66.61 ms per query instead of 989.62 ms for an ensemble method with 16 prompts. If this holds, the paper's collaborative-condensation view replaces the field's reliance on retrieving one ideal prompt, and makes multi-prompt VICL practical.","feed_headline":"Condensing 16 prompts beats pick-one visual in-context learning","feed_subtitle":"A lightweight plugin fuses several prompt candidates into one, improving three vision tasks while avoiding output ensembling.","key_machinery":"Condenser is a lightweight external module whose core operation is patch-wise cross-attention: for a query patch at position $(h,w)$, attention is restricted to prompt patches at that same $(h,w)$, with self-attention first applied to prompt images and labels so spatial correspondence is preserved. That same-position restriction is what lets the module fuse $K$ prompts into one resolution-preserving prompt while avoiding the drift full cross-attention introduces, and the authors show full attention actually hurts. The module is trained end-to-end with token prediction loss (cross-entropy over VQGAN codebook tokens for the masked query label) plus a cosine pre-alignment loss pulling the condensed image and label features toward the query's.","core_discovery":"The central claim is that combining multiple candidate prompts at the input level—rather than selecting the best one or averaging separate outputs—makes visual in-context learning both more accurate and more efficient. Condenser treats each prompt as an image-label pair, locates informative patches with self-attention, and uses patch-wise cross-attention to let each query patch gather evidence only from the same spatial position across all candidate prompts. The condensed image and label features are placed on the standard inpainting canvas with the query, and Condenser is trained end-to-end with token prediction and pre-alignment losses while the MAE-VQGAN backbone stays frozen. The authors report consistent gains over single-prompt selection, voting ensembles, and PEFT baselines on segmentation, detection, and colorization, with accuracy improving as the number of prompts grows from 1 to 32.","pith_inferences":["A testable extension implied by the patch-wise restriction: translate or crop-shift a prompt's object within its frame and measure the mIoU drop; if it is large, Condenser's success is tied to spatial alignment rather than semantic context.","The same condensation layer could be plugged into other token-based vision backbones without retraining them, since it only exchanges patch features and consumes frozen-backbone feedback through the token prediction loss.","Because the pre-alignment loss is reported as a regularizer rather than a full learning signal, a backbone-agnostic zero-shot variant would need a different semantic supervision source, for example distilling from a teacher model.","The plateau from K=16 to K=32 (46.63 to 46.73) suggests returns diminish; a future method may need to select or prune prompts before condensation."],"forward_implications":["Accuracy keeps climbing with more prompts: Condenser segmentation mIoU rises from 44.14 at K=1 to 46.73 at K=32, with inference time growing only from 59.17 ms to 74.13 ms per query.","Condenser beats the strongest single-prompt baseline by 8.09% relative on segmentation and 3.31% on detection, and lowers colorization MSE to 0.54 from 0.58.","Output ensembling (voting) is harder to design and far slower: Prompt-SelF needs 989.62 ms per query for 16 prompts, while Condenser uses 66.61 ms.","Condenser narrows the gap between weak and strong retrievers, so even random or unsupervised prompt retrieval becomes more usable in multi-prompt mode."],"supporting_citations":[{"why":"Supplies the frozen MAE-VQGAN inpainting backbone and the canvas construction that Condenser plugs into.","marker":"[2]"},{"why":"Provides the Prompt-SelF retrieval and output-ensembling baseline, plus the pixel-level retrieval used to gather candidates.","marker":"[24]"},{"why":"Provides the Partial2Global single-prompt selection and voting baselines Condenser must beat.","marker":"[34]"},{"why":"Provides the InMeMo pixel-instruction baseline that is Condenser's strongest single-prompt competitor on segmentation.","marker":"[36]"},{"why":"Provides the VPR retriever baselines (UnsupPR/SupPR) and establishes the single-prompt selection paradigm Condenser argues against.","marker":"[37]"},{"why":"Provides the VQGAN encoder/tokenizer whose codebook tokens define the token prediction loss.","marker":"[7]"},{"why":"Provides the masked autoencoder backbone used inside MAE-VQGAN for token prediction.","marker":"[10]"},{"why":"Defines the Pascal-5i dataset and evaluation protocol for the main segmentation experiments.","marker":"[21]"}],"fun_headline_variants":["Collaborative prompts beat competitive pick in visual in-context learning","Condense multiple prompts, not pick one: wins visual in-context learning","Condenser fuses prompts to sharpen visual in-context learning","Prompt collaboration condenses visual in-context learning, boosts accuracy","Forget picking: condense prompts to win visual in-context learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the useful evidence in a prompt always sits at the same spatial position as the query patch it informs; prompts whose objects or annotations are shifted or scaled would break this assumption and cap Condenser's generalization.","fun_headline_variants_meta":{"raw":{"variants":["Collaborative prompts beat competitive pick in visual in-context learning","Condense multiple prompts, not pick one: wins visual in-context learning","Condenser fuses prompts to sharpen visual in-context learning","Prompt collaboration condenses visual in-context learning, boosts accuracy","Forget picking: condense prompts to win visual in-context learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000822,"raw_usage":{"total_tokens":3582,"prompt_tokens":917,"completion_tokens":2665,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":2574}},"tokens_in":533,"tokens_out":2665,"duration_ms":19509,"temperature":1.0,"reasoning_tokens":2574,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:08:23.144982+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained Condenser and evaluate it on Pascal-5i after applying a random spatial shift of 16-32 pixels to the prompt images and labels at test time. A human can still map the shifted prompt to the query, but Condenser cannot see cross-position matches; a large accuracy drop relative to unshifted prompts would show the same-position assumption is load-bearing.","supporting_citations":[{"cited_title":"Visual prompting via image inpaint- ing","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen MAE-VQGAN inpainting backbone and the canvas construction that Condenser plugs into."},{"cited_title":"Towards global optimal visual in-context learning prompt selection","cited_arxiv_id":null,"evidence_quote":"Provides the Partial2Global single-prompt selection and voting baselines Condenser must beat."},{"cited_title":"Instruct me more! random prompt- ing for visual in-context learning","cited_arxiv_id":null,"evidence_quote":"Provides the InMeMo pixel-instruction baseline that is Condenser's strongest single-prompt competitor on segmentation."},{"cited_title":"What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems, 36, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the VPR retriever baselines (UnsupPR/SupPR) and establishes the single-prompt selection paradigm Condenser argues against."},{"cited_title":"Taming transformers for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Provides the VQGAN encoder/tokenizer whose codebook tokens define the token prediction loss."}],"review_version":1}