{"id":"60f4fadc-07bf-4ff6-9557-fc6d38f31602","arxiv_id":"2505.19455","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MM-Prompt couples the visual and language prompt paths in continual VQA, and reports higher average accuracy and lower forgetting than existing prompt-based methods.","lead":"This paper proposes MM-Prompt, a method for continual visual question answering that mixes cross-modal signals into prompt selection and reconstructs masked prompts through intra- and inter-modal attention. It reports improved accuracy and lower forgetting than nine baselines on VQA v2 and NExT-QA.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'consistently outperforms' claim rests on single-run Table 1 numbers with hyperparameters tuned on the same benchmark; no error bars or held-out validation are reported, so the ~1-point margins are not established as real.","rationale":"I read the paper as making two coupled claims: (i) MM-Prompt outperforms prior prompt-based methods in continual VQA, and (ii) the mechanism is reduced modality imbalance. The reader tagged the modality-balance diagnostics as the weakest assumption; I agree those diagnostics are under-specified, but I think the more load-bearing weakness is that the headline comparison in Table 1 has no variance estimates and no described validation protocol. Since the margins are small in several settings and hyperparameters are visibly tuned on the reported benchmark, the empirical superiority could be an artifact of selection noise. This does not mean the method is wrong; the ablations and the consistent direction of the gains are suggestive. But a single run cannot support a 'consistently outperforms' claim at this margin scale. I therefore keep the reader's conditional verdict: the paper should be accepted only after multi-seed results with a pre-registered validation split, and the modality-balance metrics should also be grounded by specifying the V-only/Q-only evaluation and testing them on a deliberately biased baseline. Agreement is partial because the reader's rationale already lists missing error bars and validation, although their stated weakest assumption focuses on the diagnostic metrics rather than the primary comparison.","tokens_in":17295,"tokens_out":11708,"duration_ms":130959,"concrete_test":"Re-run all Table 1 methods with 10 independent seeds and a pre-registered protocol: choose the mask ratio, alignment weights, and prompt counts on a held-out validation split (e.g., a separate question-type or object-class group) before computing test accuracy, and report mean plus/minus standard deviation plus the smallest pairwise margin in each of the six settings. If the VQA v2 DI margin (36.223 vs 35.187) or any smaller margin is within one pooled standard error, or if the ranking changes across seeds, the 'consistently outperforms' claim is not established. As a secondary check within the same runs, construct vision-only and question-only inputs by masking the absent modality and confirm that Eqs. 12-13 rank a deliberately language-biased baseline below MM-Prompt before using those metrics as mechanistic evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is Section 4.2's statement that MM-Prompt 'consistently outperforms all the other methods across all settings' (Table 1). The evidence for that claim is single-run: no standard deviation, seed count, or significance test appears anywhere in the paper. Hyperparameter choices are made on the benchmark being reported: Fig. 4 sweeps the mask ratio and the inter-modal alignment weight on VQA v2 DI, Fig. 9 sweeps the intra-recovery weight, and Tables 9-11 select prompt counts and loss variants on the same setting, with no held-out validation split described. In VQA v2 DI, the margin over the best baseline is about 1 point (36.223 vs 35.187), and other settings have margins of a similar order. Without variance estimates, this margin is not distinguishable from run-to-run noise, so 'consistently' is unsupported. The modality-balance mechanism does not rescue this: Appendix B.1's metrics (Eqs. 12-13) are computed from the same runs and are never validated; the paper does not specify how a vision-only or question-only input is constructed for VQA, and no ablation controls for the extra prompt capacity and extra loss terms. The central claim therefore rests on an uncontrolled single-run comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MM-Prompt, a prompt-tuning method for continual Visual Question Answering (CVQA). The method introduces cross-modal prompt query, which mixes features from the opposite modality before prompt retrieval, and cross-modal prompt recovery, which applies identical random masks to visual and textual prompts and reconstructs them through intra-modal and inter-modal recovery stages with alignment losses. The authors evaluate on VQA v2 and NExT-QA under Question Increment, Class Increment, and Dual Increment settings, comparing against nine baselines. They report that MM-Prompt achieves higher average accuracy and lower forgetting than all baselines across all settings, and they present diagnostic metrics and ablations intended to show that the method reduces modality imbalance.","tokens_in":17609,"tokens_out":6209,"duration_ms":56491,"significance":"Continual VQA is an active and practically relevant problem, and the paper identifies a plausible failure mode in existing prompt-based methods: cross-modal prompt isolation may reinforce language-dominant modality bias. If the reported gains are real, MM-Prompt would be a useful contribution, and the paper is strengthened by releasing code, evaluating on two datasets and three incremental settings, and providing component ablations. The main limitation is that the central claim of consistent superiority currently rests on single-run numbers with hyperparameters selected on the reported benchmark, so the empirical evidence has not yet established the claim.","major_comments":[{"comment":"The central claim that \"MM-Prompt consistently outperforms all the other methods across all settings\" is not supported by the evidence as reported. Every number in Table 1 appears to come from a single run, and the paper reports no standard deviations, no number of seeds, and no significance tests. The margins over the best baseline are around one point in several settings, for example 36.223 versus 35.187 for MaPLe on VQA v2 DI and 16.757 versus 15.525 for VQACL on VQA v2 QI. Without run-to-run variance information, these margins are not distinguishable from noise. Reporting multiple seeds, error bars, or pairwise significance tests is necessary before the \"consistently\" claim can be accepted.","section":"Section 4.2, Table 1"},{"comment":"The hyperparameters are selected on the same benchmark used for the headline results. Figure 4 sweeps the mask ratio delta and the inter-modal alignment weight alpha on VQA v2 DI; Figure 9 sweeps the intra-recovery weight beta on the same setting; and Tables 9, 10, and 11 select prompt counts, recovery phases, and loss variants using VQA v2 DI. No separate validation split is described. This creates a risk of selection bias, because the reported gains may reflect tuned hyperparameters rather than a robust advantage. The authors should either use a held-out validation split for hyperparameter selection or show that the chosen values perform comparably across all reported settings.","section":"Section 4.1 and Appendix B"},{"comment":"The modality-balance diagnostics used to support the mechanism claim are not validated and are not defined precisely enough to be reproduced. The paper does not specify how a vision-only or question-only input is constructed for VQA, which is nontrivial because the decoder presumably still requires both an image and a question to produce an answer. Moreover, the paper does not establish a causal link between improvement on these custom metrics and the accuracy gains: the ablation in Table 3 does not control for the extra loss terms or additional prompt capacity introduced by the recovery module. The authors should specify the input construction, validate the diagnostics on a known imbalance case, and include controlled ablations that separate the effect of the alignment losses from the effect of the extra parameters.","section":"Appendix B.1, Eqs. (12)-(13)"},{"comment":"The comparison omits at least two prior CVQA methods discussed in the paper. Section 2 cites Symbolic Replay [8] as a prompt-based CVQA method and also mentions the modality-aware feature distillation approach [5], but Section 4.2 compares only six general continual-learning methods and three CVQA methods ([4], [6], and [9]). If these cited methods are applicable to the same experimental protocol, their absence weakens the claim that MM-Prompt \"surpasses prior approaches.\" Adding these baselines, or explicitly explaining why they are not comparable, would substantially strengthen the paper.","section":"Section 2 and Section 4.2"}],"minor_comments":[{"comment":"The naming is inconsistent: \"Cross-Modal Prompts Query\" and \"Cross-Modal Prompt Query\" (also \"Recovery\" vs \"Recoveries\") are used interchangeably. Please standardize the terminology.","section":"Throughout"},{"comment":"The phrase \"vice verse\" should be \"vice versa.\"","section":"Section 3.3, Eq. (10)"},{"comment":"The sentence \"these results reveal the complementary neatural of our two components\" contains a typo; it should presumably read \"complementary nature.\"","section":"Section 4.4"},{"comment":"The table is extremely hard to read because large blocks of numbers are run together without clear column separation. The table would benefit from spacing, vertical rules, or separate sub-tables for VQA v2 and NExT-QA.","section":"Table 1"},{"comment":"The vertical axis label \"Acc After Normalized\" is grammatically incomplete; it should be \"Normalized Accuracy\" or \"Accuracy After Normalization.\"","section":"Figure 8"},{"comment":"Reference [7] appears to be a duplicate of Reference [1] (REVIVE) and should be removed or replaced.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and the proposed components are reasonable, but the empirical support needs strengthening before acceptance. The main blockers are the absence of any variance or significance information and the selection of hyperparameters on the reported benchmark without a validation split. The missing cited baselines should also be addressed. These issues appear fixable within the scope of a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"MM-Prompt is a sensible prompt-tuning framework for continual VQA, and the paper does more ablation work than most. But the central claim — 'consistently outperforms all the other methods across all settings' — is not backed by any variance estimate. Every number in Table 1 is a single run, with no seeds, no standard deviations, no significance tests. The VQA v2 DI margin over MaPLe is about 1 point (36.223 vs 35.187), which is within run-to-run noise for this kind of model. The larger margins in CI and QI (roughly 1.8 and 1.2 points) are more encouraging, but still unquantified. So treat the headline as provisional.\n\nWhat is actually new: the cross-modal prompt query, which mixes the opposite modality into each query before retrieval, and the shared masking plus hierarchical intra/inter recovery. Neither is exotic — the pieces are cross-attention, MAE-style reconstruction, and alignment losses — but the specific combination for CVQA is new, and the ablations show each piece contributes. The paper also includes a comparison of fusion strategies at both query and injection stages, hyperparameter sweeps for the three loss weights, and an honest limitations paragraph about random masking. Code is public. That is real work.\n\nThe soft spots are real but not fatal. The hypers (δ, α, β, prompt counts) are tuned on VQA v2 DI, the same benchmark that produces the headline numbers, with no held-out validation split. That likely inflates the results slightly. Two cited CVQA baselines — Symbolic Replay and the modality-aware distillation paper — are absent from Table 1; they should be included or explicitly excluded with a reason. And the modality-balance diagnostics in Appendix B.1 are under-specified: the paper defines Modality Merge Effectiveness and Modality Difference but never says how a vision-only or question-only input is constructed for a VQA model. Without that, the link between the losses and balanced engagement is suggestive, not established.\n\nThe paper is coherent and the method is plausible. It deserves a serious referee, but the referee should require multi-seed runs with variance, a proper validation split for hyperparameters, and a description of the modality-dropout protocol. I would not cite it in my own work (not my area), but I'd bring it to a reading group to discuss the validation gap, and I'd accept a review assignment on it.","headline":"Plausible, well-ablated prompt-tuning method for continual VQA, but the 'consistently outperforms' claim rests on single-run numbers; worth peer review with a demand for variance and cleaner validation.","tokens_in":18084,"tokens_out":3156,"would_cite":false,"duration_ms":29150,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MM-Prompt outperforms prior prompt-based methods in continual visual question answering by making prompt selection and injection explicitly cross-modal.","keywords":["continual learning","visual question answering","prompt tuning","modality imbalance","cross-modal interaction","catastrophic forgetting","multi-modal learning","pre-trained models"],"falsifier":"Run MM-Prompt and a baseline with the same number of prompts and the same auxiliary losses but without cross-modal prompt query and recovery; if the baseline matches MM-Prompt's accuracy and forgetting on VQA v2 under the DI setting, the cross-modal mechanism is not the cause. Also, evaluate both models with only vision or only question input and check whether MM-Prompt's vision-only accuracy approaches its question-only accuracy; if the gap remains large, the balanced-engagement claim is contradicted.","tokens_in":17079,"feed_emoji":"🖼️","tokens_out":5803,"duration_ms":50545,"temperature":0.7,"pith_summary":"Continual visual question answering (CVQA) asks a model to keep answering questions about images as new question types and object classes arrive over time, without forgetting old ones. This paper argues that prompt-based CVQA methods fail because they build visual and textual prompts in isolation, which feeds the language bias already present in VQA and worsens as tasks accumulate. The paper proposes MM-Prompt, which injects cross-modal information at two points: when selecting prompts and when recovering them after masking. The authors report that MM-Prompt beats nine prior methods in accuracy and forgetting on VQA v2 and NExT-QA across question, class, and dual increment settings, and that its gains align with better modality balance. If correct, the work identifies a concrete fix for a known failure mode and offers a diagnostic for measuring modality engagement.","feed_headline":"MM-Prompt beats nine prior methods in continual visual QA","feed_subtitle":"It forces vision and question prompts to interact during selection and recovery, slowing forgetting.","key_machinery":"The two load-bearing mechanisms are cross-modal prompt query and cross-modal prompt recovery. Query mixes the opposite modality's signal into each retrieval query before top-k key matching, with residuals and modulation weights preserving each modality's identity. Recovery applies the same random mask to both selected prompts, reconstructs them intra-modally (self-attention with a small cross-modal term, plus an orthogonality-regularized residual matrix), then inter-modally (cross-attention followed by gated refinement), and pulls the two recovered prompts into directional agreement with a cosine alignment loss. Together they ensure that the prompts injected into the frozen encoder carry joint cross-modal information rather than isolated unimodal cues.","core_discovery":"The central claim is that the degradation of prompt-based CVQA comes from cross-modal prompt isolation, and that making both the query and the recovery stages cross-modal removes that degradation. MM-Prompt's cross-modal prompt query forms each modality's retrieval query by attending to the opposite modality's features, then blending the result with the original features through learnable modulation weights, so prompt selection is guided by joint semantics rather than unimodal bias. Its cross-modal prompt recovery applies one shared random binary mask to both the visual and question prompts, reconstructs the masked entries first within each modality using self-attention plus a light cross-modal term, and then across modalities using cross-attention, a gated refinement block, and a cosine alignment loss that keeps the two recovered prompts directionally consistent. The paper reports consistent wins over all compared methods in average accuracy and inter-task forgetting, with its own modality-difference and modality-merge metrics moving in the expected directions.","pith_inferences":["The editor's inference: the same mechanism may transfer to video-based continual QA, where temporal and language biases interact differently, but the optimal masking ratio and alignment weights would need per-dataset tuning.","A testable extension is to replace the random mask with an attention-based or task-aware mask, which the paper itself lists as future work.","The modality-difference metric could serve as a general diagnostic for detecting when a continual multimodal model is memorizing language patterns instead of grounding in images.","If the coreset-free version holds, MM-Prompt offers a lightweight alternative to replay-based continual multimodal methods."],"forward_implications":["Prompt-based CVQA should treat vision and language prompt selection as a joint problem rather than two independent retrievals.","The same cross-modal query-and-recovery pattern could be applied to other continual multimodal tasks where one modality dominates.","Modality-balance diagnostics like Eqs. (12) and (13) provide a way to verify why a continual multimodal method improves, beyond aggregate accuracy.","A model with balanced engagement should be less sensitive to language priors, making it more likely to answer based on the actual image content.","The reduced forgetting without extra memory suggests the approach is compatible with memory-free or small-memory deployment."],"supporting_citations":[{"why":"Defines the CVQA problem setting, the incremental task splits, and provides the VQACL baseline that MM-Prompt must beat.","marker":"[4]"},{"why":"Supplies the dual general/expert prompt structure and the DualPrompt baseline; also informs the insertion-layer design.","marker":"[10]"},{"why":"Provides the L2P prompt-selection method and the query-key alignment loss used in the training objective.","marker":"[11]"},{"why":"Introduces the Triplet fusion prompt pool baseline that MM-Prompt compares against.","marker":"[6]"},{"why":"Provides the MAPLE multi-modal prompt learning baseline, the strongest non-CVQA competitor in the results.","marker":"[17]"},{"why":"Introduces the CluMo cluster-based modality fusion baseline for continual VQA.","marker":"[9]"},{"why":"Supplies evidence that language tends to dominate in VQA, the premise behind the modality-imbalance argument.","marker":"[18]"},{"why":"Provides the masked-autoencoder inspiration for the shared-mask recovery mechanism.","marker":"[40]"}],"fun_headline_variants":["Cross-modal prompts beat isolated ones in continual VQA","MM-Prompt: linking vision and text prompts cuts forgetting","Fusing prompt selection and recovery improves VQA stability","Cross-modal query and recovery keep VQA prompts in balance","MM-Prompt pairs prompts across modalities to retain knowledge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's central claim depends on the assumption that its diagnostic metrics—accuracy measured with only vision input or only question input—actually capture modality engagement, and that the accuracy gains come from the measured balance rather than from the added losses, extra prompt capacity, or hyperparameter tuning.","fun_headline_variants_meta":{"raw":{"variants":["Cross-modal prompts beat isolated ones in continual VQA","MM-Prompt: linking vision and text prompts cuts forgetting","Fusing prompt selection and recovery improves VQA stability","Cross-modal query and recovery keep VQA prompts in balance","MM-Prompt pairs prompts across modalities to retain knowledge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000154,"raw_usage":{"total_tokens":1165,"prompt_tokens":852,"completion_tokens":313,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":235}},"tokens_in":468,"tokens_out":313,"duration_ms":3870,"temperature":1.0,"reasoning_tokens":235,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:13:45.343468+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MM-Prompt and a baseline with the same number of prompts and the same auxiliary losses but without cross-modal prompt query and recovery; if the baseline matches MM-Prompt's accuracy and forgetting on VQA v2 under the DI setting, the cross-modal mechanism is not the cause. Also, evaluate both models with only vision or only question input and check whether MM-Prompt's vision-only accuracy approaches its question-only accuracy; if the gap remains large, the balanced-engagement claim is contradicted.","supporting_citations":[{"cited_title":"Vqacl: A novel visual question answering continual learning setting","cited_arxiv_id":null,"evidence_quote":"Defines the CVQA problem setting, the incremental task splits, and provides the VQACL baseline that MM-Prompt must beat."},{"cited_title":"DualPrompt: Complementary prompting for rehearsal-free continual learning","cited_arxiv_id":null,"evidence_quote":"Supplies the dual general/expert prompt structure and the DualPrompt baseline; also informs the insertion-layer design."},{"cited_title":"Learning to prompt for continual learning","cited_arxiv_id":null,"evidence_quote":"Provides the L2P prompt-selection method and the query-key alignment loss used in the training objective."},{"cited_title":"Decouple before interact: Multi-modal prompt learning for continual visual question answering","cited_arxiv_id":null,"evidence_quote":"Introduces the Triplet fusion prompt pool baseline that MM-Prompt compares against."},{"cited_title":"MAPLE: Multi-modal prompt learning","cited_arxiv_id":null,"evidence_quote":"Provides the MAPLE multi-modal prompt learning baseline, the strongest non-CVQA competitor in the results."},{"cited_title":"CluMo: Cluster-based Modality Fusion Prompt for Continual Learning in Visual Question Answering","cited_arxiv_id":"2408.11742","evidence_quote":"Introduces the CluMo cluster-based modality fusion baseline for continual VQA."},{"cited_title":"Overcoming language priors in visual question answering with adversarial regularization","cited_arxiv_id":null,"evidence_quote":"Supplies evidence that language tends to dominate in VQA, the premise behind the modality-imbalance argument."},{"cited_title":"Masked autoencoders are scalable vision learners","cited_arxiv_id":null,"evidence_quote":"Provides the masked-autoencoder inspiration for the shared-mask recovery mechanism."}],"review_version":1}