{"id":"af38c35a-3b11-4d20-97c3-3590acbcc7eb","arxiv_id":"2506.10550","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CR-CLIP adds cross-modal attention and multi-scale test augmentation to an AVION baseline, reporting 66.8% mAP and 82.1% nDCG, first place on EPIC-KITCHENS-100 multi-instance retrieval 2025.","lead":"This report adds cross-modal attention between video and text encoders on top of the AVION/SMSLoss pipeline and reports 66.8% mAP and 82.1% nDCG on the EPIC-KITCHENS-100 multi-instance retrieval leaderboard. The result is a first-place challenge entry, but the report omits key details and contains a dimensionally inconsistent equation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CMCR's claimed gain is not causally established because the SMSLoss baseline in Table 1 is not shown to be trained under the same protocol; the reported +4.1 mAP may reflect pipeline differences rather than the module.","rationale":"The reader's strongest concern is exactly the causal attribution of the empirical gain: the SMSLoss baseline may not be trained under identical conditions, so the reported improvement could come from protocol differences. I agree with this as the load-bearing issue. The paper's own evidence is internally consistent in the ablation table, and the numbers are plausible for a challenge report, but the missing baseline protocol, the dimension mismatch in Eq. (5), and the absence of released code prevent full verification. My read does not move the verdict: CONDITIONAL remains appropriate because the concerns are addressable with a re-run and a code release, and the core architecture is a reasonable extension. No stronger objection, such as an internal contradiction that would make the method impossible, is present in the manuscript.","tokens_in":5203,"tokens_out":2728,"duration_ms":35660,"concrete_test":"Reproduce the SMSLoss baseline inside the authors' own codebase by removing only the CMCR module (keeping the same AVION initialization, SMSLoss, optimizer settings, batch size 160, frame sampling, and TTA protocol), then compare its mAP/nDCG to the CR-CLIP row. If the gap is not close to the reported +4.1 mAP / +7.9 nDCG, the improvement cannot be attributed to CMCR. A second check is to publish the exact training script and hyperparameters for both the baseline and CR-CLIP runs so the comparison is reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the Cross-Modal Context Refinement (CMCR) module causes a 4.1 mAP and 7.9 nDCG improvement over SMSLoss. This causal attribution depends on Table 1's SMSLoss row being a like-for-like baseline: same AVION initialization, same frame sampling and resolution, same batch size, same optimizer schedule, same epochs, and same evaluation. Section 3.1 only states CR-CLIP is fine-tuned from a pretrained model built from 'SMSLoss and AVION' and uses 'the same training setup as SMSLoss', but it never reports the number of fine-tuning epochs, the frame sampling details, or whether the SMSLoss baseline row was produced by re-running that protocol or taken from the prior SMSLoss paper. The ambiguity is compounded by the introduction comparing against a different '2024 ensemble' baseline (63.8 mAP, 74.3 nDCG) rather than the Table 1 SMSLoss row (62.1 mAP, 73.0 nDCG), making it unclear which comparison supports the central claim. If the baseline came from a different experimental pipeline, the +4.1 mAP could be partly or entirely due to data sampling, training length, or evaluation details rather than CMCR. Additionally, Eq. (5) has a dimension inconsistency: F_t ∈ R^{B×Dt} is added to F_{v←t}, whose dimension is Da after projecting Q_t to a common attention dimension; the residual only aligns if Dt = Da, which is not stated. While this does not directly invalidate the leaderboard result, it means the method is not precisely specified enough to reproduce from the report alone. Together, the missing baseline protocol and incomplete method specification leave the central causal claim under-supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents ContextRefine-CLIP (CR-CLIP), a dual-encoder model designed for the EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2025. The method extends the AVION architecture with a Cross-Modal Context Refinement (CMCR) module that performs bidirectional cross-attention between visual and textual features, followed by a Gated Feed-Forward Network (GatedFFN) and LayerNorm residual blocks. Training uses the Symmetric Multi-Similarity Loss on a pretrained checkpoint built from SMSLoss and AVION. At inference, features are aggregated under horizontal flipping and multi-scale resizing (0.875x, 1.0x, 1.125x). The authors report a single-model, non-ensemble result of 66.78 mAP and 82.08 nDCG on the public leaderboard, stating that it achieves 1st place. Table 1 shows an ablation from SMSLoss to CR-CLIP (+4.1 mAP, +7.9 nDCG) and further small increments from GatedFFN, flip, and multi-scale test-time augmentation.","tokens_in":5543,"tokens_out":4680,"duration_ms":51635,"significance":"If the reported leaderboard numbers hold, this is a strong empirical result for the EK-100 MIR benchmark and demonstrates that cross-modal attention refinement before projection can improve a CLIP-style dual encoder for multi-instance retrieval with soft labels. The paper provides a clear ablation chain, reports an external leaderboard result, and promises code release, which are concrete strengths. However, the scientific contribution is primarily empirical and incremental; there is no theoretical analysis, and the novelty relative to existing cross-attention fusion designs is limited. The main value is in validating the CMCR module and the enhanced TTA strategy on a challenging benchmark, provided the baseline comparison is like-for-like.","major_comments":[{"comment":"The causal attribution of the reported gains to the CMCR module requires that the SMSLoss row in Table 1 is a like-for-like baseline produced under the identical protocol as CR-CLIP (same pretrained initialization, same frame sampling, same batch composition, same training length, same evaluation). Section 3.1 only says 'the same training setup as SMSLoss' and does not state whether this baseline was re-run by the authors or reused from the prior SMSLoss paper. This ambiguity is compounded by the Introduction comparing CR-CLIP to a different '2024 ensemble' baseline (63.8 mAP, 74.3 nDCG) rather than the Table 1 SMSLoss row (62.1 mAP, 73.0 nDCG). Please specify exactly which baseline is the control for each claim and report the protocol details (epochs, frames, resolution, batch composition) for both the baseline and the proposed method.","section":"Section 3.2, Table 1 vs. Introduction"},{"comment":"As written, Eq. (5) is dimensionally inconsistent: F_t ∈ R^{B×Dt} is added to F_{v←t}, but F_{v←t} is the output of the multi-head attention with query Q_t ∈ R^{B×Da} and keys/values V_v ∈ R^{B×Da}, so the residual requires D_t = D_a, which is never stated. If the intended residual is on the projected query Q_t, or if the architecture sets D_t = D_a by construction, please rewrite the notation to make the dimensions explicit. As it stands, the module is not precisely specified enough to reproduce from the equations.","section":"Section 2.2, Eq. (5)"},{"comment":"The per-component ablation increments are small (e.g., +0.2 mAP from 'w/Flip' to 'w/Flip+Scale' and a drop of 0.6 mAP in V2T from 'CR-CLIP' to 'w/GatedFFN'), yet the text claims these components play 'a crucial role.' Without error bars, multiple seeds, or significance tests, these small differences cannot be distinguished from training noise. Please report variance or at least state that these increments are single-run observations.","section":"Section 3.2, Table 1"},{"comment":"The GatedFFN is not defined: Eq. (6) refers to GatedFFN(F'_v) but provides no equation or description of the gating mechanism, the dimensionality, the activation functions, or how it differs from a standard FFN. The symmetric vision-guided text refinement operation is also not shown. This prevents reproduction of the method independent of the promised code release.","section":"Section 2.2, Eq. (6)"}],"minor_comments":[{"comment":"The heading 'Experimemts' contains a typo and should read 'Experiments.'","section":"Section 3"},{"comment":"The abstract reports 66.78 mAP and 82.08 nDCG, while Table 1 reports 66.8 and 82.1; the phrase 'retained the first three valid digits without rounding' is ambiguous and should be clarified (e.g., significant figures vs. decimal places).","section":"Abstract vs. Table 1"},{"comment":"The multi-scale augmentation '0.875x, 1.0x and 1.125x of the original resolution' should specify the reference resolution for the base case and how center cropping interacts with the original aspect ratio.","section":"Section 2.4"},{"comment":"Reference [9] appears unrelated to the sentence citing it on contrastive representation learning; please verify the citation placement and relevance.","section":"References"},{"comment":"Eq. (7) defines only the difference R = C_ij - C_ik and does not provide the SMSLoss formula or define the role of the relaxation factor τ; add the full loss expression or a precise citation to make the objective reproducible.","section":"Section 2.3"}],"recommendation":"major_revision","confidential_remarks":"This is a challenge report rather than a full research paper, and its contribution is empirical. The main risk is that the central improvement over SMSLoss is not causally established without a like-for-like baseline; the authors must clarify the baseline protocol and reconcile the two different baseline numbers used in the Introduction and Table 1. The dimension inconsistency in Eq. (5) also needs to be fixed. If the authors provide these clarifications, the paper could be acceptable as a workshop-style contribution, but its scope may be limited for a top-tier journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid competition report with a sensible incremental idea, but the headline '+4.1 mAP from CMCR' is not actually pinned down by the experiments as written. The ablation trend is plausible and the leaderboard result is real, but the missing protocol details for the SMSLoss baseline and a dimension mismatch in Eq. (5) keep the central attribution from being fully supported.\n\nWhat's actually new: inserting a lightweight cross-attention module (CMCR) between the encoders and the projection heads of AVION, plus GatedFFN and multi-scale TTA, and showing consistent increments on EK-100 multi-instance retrieval. That's a reasonable extension of known parts. The ablation table is monotonic (SMSLoss -> +CMCR -> +GatedFFN -> +Flip -> +Scale), which is a good sign. The paper is honest about its limitations: no ensembling, no overhead analysis, only ViT-L. That counts for something. The code is promised, not yet available.\n\nSoft spots: the biggest is the baseline comparability. Table 1's SMSLoss row is the comparison that matters, but the text never says whether that row was produced by re-running the SMSLoss protocol under identical initialization, sampling, frames, and evaluation, or copied from the prior paper. The intro instead compares against a 2024 ensemble baseline, which is a different target. Without that control, the +4.1 mAP could be partly pipeline differences. That's a real gap, though fixable by a careful re-run or a clear statement. Second, Eq. (5) adds F_t to the cross-attention output which lives in attention dimension D_a; the residual only makes sense if D_t = D_a, which is never stated. That is a specification bug that matters for reproducibility. Minor: no error bars or significance tests; typical for challenge reports, so I wouldn't weight it heavily.\n\nThe stress-test note is on target. I agree with the reader's conditional verdict. The central idea is defensible; the evidence is just incomplete, not contradictory.\n\nBottom line: if you care about EK-100 retrieval or CLIP-style video-text dual encoders, this is a useful data point. It deserves a serious referee (workshop or short paper) because the result is plausible and the ablations are informative, but it should be sent back for a baseline-provenance statement, a corrected Eq. (5), and ideally released code before acceptance.","headline":"Reasonable incremental architecture with a believable ablation, but the causal claim needs a proper baseline control and a fix to Eq. (5) before I'd trust the +4.1 mAP.","tokens_in":6087,"tokens_out":2260,"would_cite":false,"duration_ms":23072,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Bidirectional cross-modal attention before projection lifts EPIC-KITCHENS-100 retrieval to 66.8 mAP and 82.1 nDCG in a single model.","keywords":["multi-instance retrieval","cross-modal retrieval","EPIC-KITCHENS-100","video-text retrieval","cross-attention","soft labels","Symmetric Multi-Similarity Loss","test-time augmentation"],"falsifier":"Retrain the SMSLoss baseline from the same AVION/Ego4D pretrained checkpoint, with the same batch size (160), learning rate (1.8e-5), ViT-L-14 backbone, frame sampling, and flip/multi-scale TTA protocol, and compare on the same EPIC-KITCHENS-100 splits; if the mAP/nDCG gap is not approximately 4.1/7.9, the causal attribution to the CMCR module fails.","tokens_in":4998,"feed_emoji":"🎬","tokens_out":6831,"duration_ms":60789,"temperature":0.7,"pith_summary":"This paper tries to establish that a relatively small architectural change to a CLIP-style dual encoder, inserting a bidirectional cross-modal attention module between the visual and text encoders and the projection heads, materially improves multi-instance retrieval when relevance comes as a soft-label matrix rather than hard binary matches. On EPIC-KITCHENS-100, the resulting ContextRefine-CLIP (CR-CLIP) raises average mAP from 62.1% to 66.8% and average nDCG from 73.0% to 82.1% over the SMSLoss baseline, with the CMCR module alone contributing 4.1 mAP and 7.9 nDCG before any test-time augmentation. Combined with horizontal-flip and multi-scale test-time augmentation, the single model reaches 66.78 mAP and 82.08 nDCG on the challenge's public leaderboard without ensemble learning. A reader should care because the result suggests that feature-level inter-modal interaction, not just a better loss function, is what drives ranking quality in soft-label retrieval.","feed_headline":"Bidirectional attention lifts retrieval to 66.8 mAP","feed_subtitle":"A single CR-CLIP model beats the 2024 ensemble baseline on EPIC-KITCHENS-100 without ensembling.","key_machinery":"The load-bearing component is the Cross-Modal Context Refinement (CMCR) module: a pair of lightweight cross-attention layers, one that lets text features attend to visual features and one that does the symmetric operation for vision-guided text refinement. Each refined feature passes through a residual connection, LayerNorm, and a Gated Feed-Forward Network before the MLP projection heads. The module's job is to make each modality's representation absorb semantic cues from the other before similarity is computed, so that fine-grained correspondences such as video actions to query verbs and object combinations to descriptions are aligned in the shared embedding space. The paper argues this is what produces the gains over a projection-only dual encoder, and it is trained with Symmetric Multi-Similarity Loss to exploit the soft-label relevance matrix.","core_discovery":"The paper's central claim is that on EPIC-KITCHENS-100 multi-instance retrieval, a dual-encoder model whose visual and textual features refine each other through cross-attention before being projected into a shared space outperforms the prior SMSLoss approach by a wide margin. In the paper's comparison table, CR-CLIP with the CMCR module reaches 66.2% average mAP and 80.9% average nDCG, surpassing the SMSLoss baseline (62.1% mAP, 73.0% nDCG), and adding GatedFFN, horizontal flip, and multi-scale test-time augmentation brings it to 66.8% average mAP and 82.1% average nDCG. The authors further report that the same single model, without ensembling, scores 66.78% mAP and 82.08% nDCG on the public leaderboard, beating the 2024 ensemble-based solution's 63.8% mAP and 74.3% nDCG.","pith_inferences":["The same bidirectional refinement idea could transfer to other soft-label retrieval tasks such as composed image retrieval or video moment retrieval, but the paper only tests it on EPIC-KITCHENS-100.","The paper's ablations combine CMCR, GatedFFN, and TTA in a chain, so the individual contribution of multi-scale augmentation versus the module itself on the final leaderboard number is not isolated; a factorial ablation would settle it.","The headline comparison in the introduction is against a 2024 ensemble solution (63.8/74.3), while Table 1's baseline is SMSLoss (62.1/73.0); readers should compare like-for-like when quoting the +3.0/+7.8 improvement.","The authors' own limitation list notes no ensembling, no computational-overhead analysis, ViT-L only, and limited hyperparameter tuning, so the reported numbers are a single configuration rather than an optimized family."],"forward_implications":["A single CR-CLIP model without ensembling reaches 66.78 mAP and 82.08 nDCG on the EPIC-KITCHENS-100 public leaderboard, above the 2024 ensemble baseline's 63.8 mAP and 74.3 nDCG.","The CMCR module alone adds 4.1 average mAP and 7.9 average nDCG over the SMSLoss baseline, so the architectural change, not just the loss or augmentation, drives most of the gain.","GatedFFN improves T2V mAP and average nDCG, and flip plus multi-scale TTA adds a further 0.6 average mAP and 1.2 average nDCG on top of CMCR.","The model trains on a single machine with 4 RTX 3090s and batch size 160, so the approach is reproducible without large compute clusters.","Because the largest relative improvement is in nDCG, the refined features mainly improve the ordering of retrieved items, not just hit-or-miss accuracy."],"supporting_citations":[{"why":"Supplies the SMSLoss objective and the pretrained model/baseline that CR-CLIP fine-tunes from; the Table 1 SMSLoss row is the direct comparison.","marker":"[8]"},{"why":"Provides the AVION dual-encoder architecture that CR-CLIP builds on.","marker":"[10]"},{"why":"Defines the CLIP dual-encoder contrastive design and the visual/text encoders used.","marker":"[7]"},{"why":"Provides the Ego4D egocentric video dataset used in pretraining.","marker":"[3]"},{"why":"The LLM-augmented Ego4D pretraining that yields the initialization for fine-tuning.","marker":"[11]"},{"why":"Defines the EPIC-KITCHENS dataset used for the retrieval task.","marker":"[1]"},{"why":"Defines the EPIC-KITCHENS-100 benchmark and its soft-label relevance matrices.","marker":"[2]"},{"why":"Motivates the Gated Feed-Forward Network component inside CMCR.","marker":"[4]"}],"fun_headline_variants":["Single model beats 2024 ensemble on EPIC-KITCHENS-100","CR-CLIP: cross-attention lifts mAP to 66.8 solo","Bidirectional refinement wins multi-instance retrieval","No ensemble needed: CR-CLIP hits 66.8 mAP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Table 1 SMSLoss baseline was trained with the same pretrained initialization, data sampling, frame selection, and evaluation protocol as CR-CLIP, so the measured 4.1 mAP and 7.9 nDCG differences can be attributed to the CMCR module; the paper does not specify this.","fun_headline_variants_meta":{"raw":{"variants":["Single model beats 2024 ensemble on EPIC-KITCHENS-100","CR-CLIP: cross-attention lifts mAP to 66.8 solo","Bidirectional refinement wins multi-instance retrieval","No ensemble needed: CR-CLIP hits 66.8 mAP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000315,"raw_usage":{"total_tokens":1783,"prompt_tokens":937,"completion_tokens":846,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":768}},"tokens_in":553,"tokens_out":846,"duration_ms":9632,"temperature":1.0,"reasoning_tokens":768,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:22:55.623311+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the SMSLoss baseline from the same AVION/Ego4D pretrained checkpoint, with the same batch size (160), learning rate (1.8e-5), ViT-L-14 backbone, frame sampling, and flip/multi-scale TTA protocol, and compare on the same EPIC-KITCHENS-100 splits; if the mAP/nDCG gap is not approximately 4.1/7.9, the causal attribution to the CMCR module fails.","supporting_citations":[{"cited_title":"Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning","cited_arxiv_id":"2402.02055","evidence_quote":"Provides the AVION dual-encoder architecture that CR-CLIP builds on."},{"cited_title":"Visual to text: Survey of image and video captioning","cited_arxiv_id":null,"evidence_quote":"Defines the CLIP dual-encoder contrastive design and the visual/text encoders used."},{"cited_title":"Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100","cited_arxiv_id":null,"evidence_quote":"Provides the Ego4D egocentric video dataset used in pretraining."},{"cited_title":"a henb \\","cited_arxiv_id":null,"evidence_quote":"The LLM-augmented Ego4D pretraining that yields the initialization for fine-tuning."},{"cited_title":"Scaling egocentric vision: The epic-kitchens dataset","cited_arxiv_id":null,"evidence_quote":"Defines the EPIC-KITCHENS-100 benchmark and its soft-label relevance matrices."}],"review_version":1}