{"id":"7df59f8a-6f63-4407-9095-7162546d6ed7","arxiv_id":"2509.17446","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MVCL-DAF++ combines prototype-based contrastive learning with coarse-to-fine attention fusion and reports improved accuracy and weighted F1 on MIntRec and MIntRec2.0.","lead":"MVCL-DAF++ adds two mechanisms to an existing multimodal intent recognition model: class prototypes for contrastive learning, and a two-stage attention fusion that combines whole-modality summaries with token-level details. On the MIntRec and MIntRec2.0 benchmarks, the upgraded model reports new state-of-the-art scores, but the evaluation compares only four baselines and reports no error bars.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Text-with-label stream may leak ground-truth labels into the classifier path at inference; the paper never states how E_tl is masked at test time, so the reported SOTA gains may be circular.","rationale":"The reader's weakest_assumption is the same load-bearing concern I identify: the manuscript does not explain how the text-with-label stream is handled at test time, so the classification path M_cf may include the ground-truth label. This is the single most important issue because the paper's headline claim is state-of-the-art accuracy on two benchmarks, and that claim is circular if E_tl leaks the label. The concern is not a stylistic preference; it is a correctness risk that can be settled by reading the code or by a simple inference-time ablation. I agree with the reader's conditional verdict: the paper should not be accepted as-is until this is resolved. Secondary issues—small baseline set, missing error bars, and the ambiguity between Table 2 and Figure 2 ablations—are real but secondary; they would weaken the strength of the empirical claim even if the leakage concern is resolved. I credit the authors for providing source code and for reporting 10-seed averages, which makes the concrete test feasible and inexpensive. No ad hominem is intended; the gap is in the written specification, not necessarily in the implementation.","tokens_in":6667,"tokens_out":4532,"duration_ms":39189,"concrete_test":"Inspect the released code at github.com/chr1s623/MVCL-DAF-PlusPlus: locate the forward pass used at inference (e.g., after checkpoint loading) and confirm whether the ground-truth label string is concatenated to text before BERT encoding to produce E_tl. Then ablate by setting E_tl = E_tm at inference only and recompute Table 1. If ACC/WF1 changes by more than the seed-level variation, the reported numbers depend on label access. If the code already masks E_tl at eval, report the exact eval-mode branch that implements this masking as the settling evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim depends on M_cf, the representation classified in Eq. (10). Section 2.1 says M_cf is built from contextualized features M_a, M_v, M_tm, and M_tl, where M_tl comes from the \"text-labeled\" encoder E_tl. The paper never states whether the label text is removed or masked at inference. If E_tl is the utterance with the true class name appended, then M_cf contains the answer at test time, and the +1.05/+4.18 WF1 gains, especially the +11.93 recall jump on MIntRec2.0, would be explained by target leakage rather than by prototype-aware alignment or coarse-to-fine fusion. Sections 2.1–2.5 and 3.2 contain no sentence describing test-time handling of E_tl, and the released code link is not accompanied by an inference-mode description. This is not an internal inconsistency—the architecture is coherent if E_tl is a training-only contrastive view—but the manuscript text is silent on exactly the point that separates a valid method from a circular one.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MVCL-DAF++, an extension of MVCL-DAF for multimodal intent recognition. The two claimed contributions are prototype-aware contrastive alignment, in which class-level prototypes are computed from mini-batch instance embeddings and used in an InfoNCE-style loss, and coarse-to-fine dynamic attention fusion, in which a 'modality-aware' transformer produces a coarse global representation that is combined with token-level features via DAF modules. The final representation M_cf is classified with cross-entropy; the total loss combines classification, prototype, and contrastive terms. Experiments are reported on MIntRec and MIntRec2.0, with accuracy, weighted F1, weighted precision, and recall averaged over 10 seeds. The paper reports new state-of-the-art numbers, with the largest gains on MIntRec2.0 (e.g., +4.18 WF1, +11.93 recall over MVCL-DAF).","tokens_in":7069,"tokens_out":2700,"duration_ms":26390,"significance":"If the reported results are valid, the contribution is a useful incremental improvement to multimodal intent recognition, especially on the long-tailed, noisy MIntRec2.0 benchmark. The manuscript has several strengths: it builds on standard external benchmarks, reuses a previously published base method, reports ablations of both proposed modules and of each loss term, and provides a public code link. The main reason the significance cannot be fully assessed from the manuscript alone is that the text-labeled stream E_tl is described without any statement of how it is handled at inference, so the central state-of-the-art claim currently rests on an unresolved leakage question. In addition, the empirical support would be materially stronger with variance or significance information, because several improvements are small relative to what one would expect from seed noise.","major_comments":[{"comment":"The text-labeled stream E_tl / M_tl is a central part of the architecture: Section 2.1 states that M_a, M_v, M_tm, and M_tl are transformed into contextualized features and fused into M_cf; Eq. (10) then classifies from M_cf. The manuscript never states how the 'text-labeled' input is created or whether the label text is removed or masked at inference. If E_tl contains the true class name or label text at test time, then the classifier path has access to the answer, and the reported gains, including the +11.93 recall improvement on MIntRec2.0, would be explained by target leakage rather than by the proposed modules. The authors must specify the exact construction of E_tl at train and test time, the masking procedure if any, and confirm whether M_cf excludes label-derived information during inference.","section":"Section 2.1, Fig. 1, Eq. (10)"},{"comment":"All reported results are averages over 10 random seeds, but no standard deviations, confidence intervals, or significance tests are provided. This is particularly important because several headline improvements are small: on MIntRec, ACC improves by +1.46 and WF1 by +1.05 over MVCL-DAF, while the ablation in Fig. 2 shows drops of roughly 1 point (76.18 to 75.17 and 75.06) when removing each module. Without variance information, the reader cannot determine whether the ablation drops or the MIntRec gains are distinguishable from seed noise. The manuscript should report per-seed spread or statistical tests for the main comparisons and ablations, or temper the strength of the claims.","section":"Table 1, Section 3.2, Fig. 2"},{"comment":"The abstract and Section 4.1 claim 'new state-of-the-art results,' but Table 1 compares the method with only four baselines (MulT, MAG-BERT, TCL-MAP, MVCL-DAF). If these are the only published strong baselines on MIntRec and MIntRec2.0, this should be stated explicitly; otherwise the 'state-of-the-art' wording overstates the evidence. The authors should either add the missing published comparisons from the MMIR literature or qualify the claim to say 'state-of-the-art among the baselines considered here.'","section":"Abstract and Table 1"}],"minor_comments":[{"comment":"The InfoNCE loss is written with a denominator over K 'j' terms, but the positive pair and the number K are not formally defined. Please define the negative set and clarify whether the sum includes the positive pair.","section":"Section 2.4, Eq. (4)"},{"comment":"The text near the top of Fig. 1 contains an unlabeled snippet ('Aww, man, then I won't to hear Jonah lecture us') that appears to be an example utterance but is not explained; either remove it or caption it as a sample input.","section":"Section 2.1 and Fig. 1"},{"comment":"The sentence 'the absence of either component reduces WF1 by average than 1.27 points' is grammatically incomplete and should be revised to report the exact average or range of drops.","section":"Section 4.2"},{"comment":"The title in the PDF body has unusual spacing 'PROTOTYPE-A W ARE' instead of 'PROTOTYPE-AWARE'; this should be corrected in the camera-ready version.","section":"Title"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper extends MVCL-DAF with two well-known ideas—prototype-based contrastive learning and coarse-to-fine attention fusion—and applies them to multimodal intent recognition. That combination is new, and the empirical work is mostly careful: two benchmarks, four metrics, ablations that drop each module, and a loss-function ablation. The reported gains are plausible if the architecture works as drawn. The code link is a plus, even if the code itself is not described in the text.\n\nThe big soft spot, and it is load-bearing, is the text-with-label stream E_tl. The architecture in Figure 1 and the equations in Section 2.1 show E_tl feeding into M_tl, which becomes part of M_cf, which is passed to the classifier in Eq. (10). The paper never states whether the label text is removed or masked at inference. If E_tl is the utterance with the true class name appended, then the +1.05 and +4.18 WF1 gains, especially the +11.93 recall jump on MIntRec2.0, are exactly what target leakage would produce. The authors may well have intended E_tl to be a training-only contrastive view, with a masked version used for classification at test time, but nothing in the text says so. That is not a minor omission; it separates a valid method from a circular one.\n\nSecondary issues are more moderate. The baseline set is thin—four methods, all older, while the paper cites newer approaches like Contextual Augmented Global Contrast (Sun et al. 2024) that are not compared. The 10-seed averages come without variances or significance tests, so the ablation drops of ~1 point could be seed noise. The attention and t-SNE analyses are qualitative and don't add much beyond the numbers.\n\nThe architecture itself is coherent, and the writing is clear even where it is silent. If the leakage concern is resolved by clarifying the test-time input stream, this is a serviceable incremental contribution for the MMIR community. As written, the state-of-the-art claim is not yet supported.\n\nRecommendation: send it to peer review, but with the explicit requirement that the authors state how E_tl is masked or omitted at inference, and ideally add a test-time-only ablation. A serious referee should engage with this; the central question is verifiable and the code is public.","headline":"Useful incremental architecture for multimodal intent recognition, but the text-with-label stream could be leaking the ground-truth label into the classifier at inference, and the paper never says how it is handled at test time.","tokens_in":7434,"tokens_out":1687,"would_cite":false,"duration_ms":15817,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding class-level prototypes plus coarse-to-fine fusion sets new records for multimodal intent recognition, with the largest rare-class gains on the long-tailed MIntRec2.0 benchmark.","keywords":["multimodal intent recognition","prototype-aware contrastive alignment","coarse-to-fine attention fusion","rare-class recognition","long-tailed distribution","dynamic attention fusion","MIntRec","representation learning"],"falsifier":"Run inference with the text-with-label stream replaced by a masked or label-free text stream, or remove that stream entirely, and measure weighted F1 on MIntRec2.0. If the +4.18 rare-class gain and the +11.93 recall gain shrink or vanish, the reported improvement depends on access to the true label at test time rather than on the two new modules.","tokens_in":6492,"feed_emoji":"🎯","tokens_out":7701,"duration_ms":59872,"temperature":0.7,"pith_summary":"Multimodal intent recognition has to decide what a user wants from speech, text, and video, and it typically struggles on rare or noisy examples. This paper tries to show that two changes fix much of that failure: anchoring contrastive learning to class-level prototypes, and fusing modality information from coarse global summaries down to fine token-level features. On the MIntRec and MIntRec2.0 benchmarks, the resulting model reports the best published accuracy and weighted F1, with the largest rare-class gains on the noisier, long-tailed MIntRec2.0 set. The authors' ablation studies are meant to show that both modules are necessary and complementary.","feed_headline":"Prototype alignment and coarse fusion lift intent recognition scores","feed_subtitle":"Class-level prototypes and hierarchical fusion beat earlier models on both intent benchmarks, especially rare classes.","key_machinery":"The central objects are class-level prototypes, \\(\\mathbf{r}_c = \\frac{1}{|I_c|} \\sum_{i \\in I_c} \\mathbf{h}_i\\), and the prototype-aware InfoNCE loss \\(\\mathcal{L}_{\\text{proto}}\\), which aligns each normalized instance embedding \\(\\mathbf{h}_i\\) to its class prototype in a temperature-scaled softmax. The second carrying mechanism is coarse-to-fine fusion: a modality-aware transformer computes \\(\\mathbf{M}_c = \\mathrm{Enc}(Q=\\mathrm{Text}, K=\\mathrm{Visual}, V=\\mathrm{Acoustic})\\), and two dynamic attention fusion modules combine \\(\\mathbf{M}_c\\) with token-level features to produce a fine representation for contrastive learning and a coarse-enhanced representation \\(\\mathbf{M}_{cf}\\) for classification. The ablation study credits these two mechanisms, jointly, for the reported accuracy and weighted-F1 gains.","core_discovery":"MVCL-DAF++ extends the MVCL-DAF architecture with two modules. First, for each class it averages the current mini-batch instance embeddings into an L2-normalized prototype, then applies a prototype-aware InfoNCE loss that pulls each instance toward its own class prototype and pushes it away from the others. Second, it computes a coarse modality summary with a modality-aware transformer encoder, then combines that summary with token-level features through two dynamic attention fusion modules, using the final coarse-enhanced representation for classification. On MIntRec the model reports 76.18 accuracy and 75.66 weighted F1, and on MIntRec2.0 it reports 60.40 accuracy and 59.23 weighted F1, improving rare-class weighted F1 by 1.05 and 4.18 points over the previous best results.","pith_inferences":["If the label-free assumption is granted, the prototype loss should also help out-of-scope detection on MIntRec2.0's open-intent samples; the paper does not report that split, so it is a natural next check.","The recall jump on MIntRec2.0 (+11.93 points) is far larger than the accuracy jump (+2.60); one plausible reading is that most of the gain comes from resolving long-tail classes, where a label-conditioned text stream would also provide the most help.","A direct test of the two modules' independence would be to add each one to a fixed base model separately and cross them on both benchmarks; the paper's ablations remove one module at a time from the full model, which does not fully separate interaction effects."],"forward_implications":["If the results hold, prototype-aware alignment offers a direct recipe for improving rare-class and long-tail performance in multimodal intent recognition, since the largest gain is on MIntRec2.0's weighted F1 (+4.18) and recall (+11.93).","The coarse-to-fine fusion's tendency to weight coarse summaries more on noisy data suggests global cross-modal semantics are safer than token-level features under noise.","Ablations imply the two modules are complementary: removing either one costs accuracy and weighted F1 on both benchmarks.","The combined objective of classification plus contrastive plus prototype losses is necessary for the best results; any two losses alone underperform the full triplet."],"supporting_citations":[{"why":"Base architecture that MVCL-DAF++ extends and the primary baseline it must beat.","marker":"[6]"},{"why":"Provides MIntRec, one of the two benchmark datasets and the source of the reported 76.18 ACC / 75.66 WF1 results.","marker":"[17]"},{"why":"Provides MIntRec2.0, the long-tailed, noisier benchmark where the largest rare-class gains (+4.18 WF1) are reported.","marker":"[18]"},{"why":"Token-level contrastive baseline that the model outperforms and that frames the contrastive objective.","marker":"[10]"},{"why":"Multimodal transformer baseline used for comparison.","marker":"[5]"},{"why":"Multimodal adaptation gate baseline used for comparison.","marker":"[13]"},{"why":"Source of the InfoNCE contrastive objective that both the multi-view and prototype losses are built on.","marker":"[12]"},{"why":"Origin of using class prototypes as semantic anchors, which the prototype-aware alignment adapts.","marker":"[16]"}],"fun_headline_variants":["Prototypes and coarse fusion lift rare-class intent F1 by 4.18","MVCL-DAF++: prototypes plus coarse fusion lift rare intent","Rare-class intent recognition improved by 4.18 WF1 with MVCL-DAF++","Prototype-aware contrastive alignment and coarse fusion aid rare classes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the text-with-label stream can be produced at inference without knowing the ground-truth intent label, so fusing it into the classifier does not leak the answer.","fun_headline_variants_meta":{"raw":{"variants":["Prototypes and coarse fusion lift rare-class intent F1 by 4.18","MVCL-DAF++: prototypes plus coarse fusion lift rare intent","Rare-class intent recognition improved by 4.18 WF1 with MVCL-DAF++","Prototype-aware contrastive alignment and coarse fusion aid rare classes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001067,"raw_usage":{"total_tokens":4436,"prompt_tokens":871,"completion_tokens":3565,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":3482}},"tokens_in":487,"tokens_out":3565,"duration_ms":21142,"temperature":1.0,"reasoning_tokens":3482,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:47:35.538803+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run inference with the text-with-label stream replaced by a masked or label-free text stream, or remove that stream entirely, and measure weighted F1 on MIntRec2.0. If the +4.18 rare-class gain and the +11.93 recall gain shrink or vanish, the reported improvement depends on access to the true label at test time rather than on the two new modules.","supporting_citations":[{"cited_title":"Mag-bert: Multimodal adapta- tion gate bert for multimodal sentiment analysis,","cited_arxiv_id":null,"evidence_quote":"Provides MIntRec2.0, the long-tailed, noisier benchmark where the largest rare-class gains (+4.18 WF1) are reported."},{"cited_title":"Multimodal transformer for un- aligned multimodal language sequences,","cited_arxiv_id":null,"evidence_quote":"Token-level contrastive baseline that the model outperforms and that frames the contrastive objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Multimodal transformer baseline used for comparison."},{"cited_title":"Multimodal transformer with multi-scale alignment for multimodal sentiment analysis,","cited_arxiv_id":null,"evidence_quote":"Source of the InfoNCE contrastive objective that both the multi-view and prototype losses are built on."}],"review_version":2}