Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MM-Prompt outperforms prior prompt-based methods in continual visual question answering by making prompt selection and injection explicitly cross-modal.

desk verdict Plausible, well-ablated prompt-tuning method for continual VQA, but the 'consistently outperforms' claim rests on single-run numbers; worth peer review with a demand for variance and cleaner validation. read the letter →

arxiv 2505.19455 v2 pith:TGK5BXU5 submitted 2025-05-26 cs.CV cs.AIcs.LG

Xu Li , Fan Lyu This is my paper
classification cs.CVcs.AIcs.LG
keywords continuallearningvisualquestionansweringprompttuningmodalityimbalancecross-modalinteractioncatastrophicforgettingmulti-modalpre-trainedmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Continual visual question answering (CVQA) asks a model to keep answering questions about images as new question types and object classes arrive over time, without forgetting old ones. This paper argues that prompt-based CVQA methods fail because they build visual and textual prompts in isolation, which feeds the language bias already present in VQA and worsens as tasks accumulate. The paper proposes MM-Prompt, which injects cross-modal information at two points: when selecting prompts and when recovering them after masking. The authors report that MM-Prompt beats nine prior methods in accuracy and forgetting on VQA v2 and NExT-QA across question, class, and dual increment settings, and that its gains align with better modality balance. If correct, the work identifies a concrete fix for a known failure mode and offers a diagnostic for measuring modality engagement.

What carries the argument

The two load-bearing mechanisms are cross-modal prompt query and cross-modal prompt recovery. Query mixes the opposite modality's signal into each retrieval query before top-k key matching, with residuals and modulation weights preserving each modality's identity. Recovery applies the same random mask to both selected prompts, reconstructs them intra-modally (self-attention with a small cross-modal term, plus an orthogonality-regularized residual matrix), then inter-modally (cross-attention followed by gated refinement), and pulls the two recovered prompts into directional agreement with a cosine alignment loss. Together they ensure that the prompts injected into the frozen encoder carry joint cross-modal information rather than isolated unimodal cues.

What would settle it

Run MM-Prompt and a baseline with the same number of prompts and the same auxiliary losses but without cross-modal prompt query and recovery; if the baseline matches MM-Prompt's accuracy and forgetting on VQA v2 under the DI setting, the cross-modal mechanism is not the cause. Also, evaluate both models with only vision or only question input and check whether MM-Prompt's vision-only accuracy approaches its question-only accuracy; if the gap remains large, the balanced-engagement claim is contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the degradation of prompt-based CVQA comes from cross-modal prompt isolation, and that making both the query and the recovery stages cross-modal removes that degradation. MM-Prompt's cross-modal prompt query forms each modality's retrieval query by attending to the opposite modality's features, then blending the result with the original features through learnable modulation weights, so prompt selection is guided by joint semantics rather than unimodal bias. Its cross-modal prompt recovery applies one shared random binary mask to both the visual and question prompts, reconstructs the masked entries first within each modality using self-attention plus a light cross-modal term, and then across modalities using cross-attention, a gated refinement block, and a cosine alignment loss that keeps the two recovered prompts directionally consistent. The paper reports consistent wins over all compared methods in average accuracy and inter-task forgetting, with its own modality-difference and modality-merge metrics moving in the expected directions.

Load-bearing premise

The paper's central claim depends on the assumption that its diagnostic metrics—accuracy measured with only vision input or only question input—actually capture modality engagement, and that the accuracy gains come from the measured balance rather than from the added losses, extra prompt capacity, or hyperparameter tuning.

Editorial extensions

If this is right

  • Prompt-based CVQA should treat vision and language prompt selection as a joint problem rather than two independent retrievals.
  • The same cross-modal query-and-recovery pattern could be applied to other continual multimodal tasks where one modality dominates.
  • Modality-balance diagnostics like Eqs. (12) and (13) provide a way to verify why a continual multimodal method improves, beyond aggregate accuracy.
  • A model with balanced engagement should be less sensitive to language priors, making it more likely to answer based on the actual image content.
  • The reduced forgetting without extra memory suggests the approach is compatible with memory-free or small-memory deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The editor's inference: the same mechanism may transfer to video-based continual QA, where temporal and language biases interact differently, but the optimal masking ratio and alignment weights would need per-dataset tuning.
  • A testable extension is to replace the random mask with an attention-based or task-aware mask, which the paper itself lists as future work.
  • The modality-difference metric could serve as a general diagnostic for detecting when a continual multimodal model is memorizing language patterns instead of grounding in images.
  • If the coreset-free version holds, MM-Prompt offers a lightweight alternative to replay-based continual multimodal methods.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MM-Prompt, a prompt-tuning method for continual Visual Question Answering (CVQA). The method introduces cross-modal prompt query, which mixes features from the opposite modality before prompt retrieval, and cross-modal prompt recovery, which applies identical random masks to visual and textual prompts and reconstructs them through intra-modal and inter-modal recovery stages with alignment losses. The authors evaluate on VQA v2 and NExT-QA under Question Increment, Class Increment, and Dual Increment settings, comparing against nine baselines. They report that MM-Prompt achieves higher average accuracy and lower forgetting than all baselines across all settings, and they present diagnostic metrics and ablations intended to show that the method reduces modality imbalance.

Significance. Continual VQA is an active and practically relevant problem, and the paper identifies a plausible failure mode in existing prompt-based methods: cross-modal prompt isolation may reinforce language-dominant modality bias. If the reported gains are real, MM-Prompt would be a useful contribution, and the paper is strengthened by releasing code, evaluating on two datasets and three incremental settings, and providing component ablations. The main limitation is that the central claim of consistent superiority currently rests on single-run numbers with hyperparameters selected on the reported benchmark, so the empirical evidence has not yet established the claim.

major comments (4)
  1. [Section 4.2, Table 1] The central claim that "MM-Prompt consistently outperforms all the other methods across all settings" is not supported by the evidence as reported. Every number in Table 1 appears to come from a single run, and the paper reports no standard deviations, no number of seeds, and no significance tests. The margins over the best baseline are around one point in several settings, for example 36.223 versus 35.187 for MaPLe on VQA v2 DI and 16.757 versus 15.525 for VQACL on VQA v2 QI. Without run-to-run variance information, these margins are not distinguishable from noise. Reporting multiple seeds, error bars, or pairwise significance tests is necessary before the "consistently" claim can be accepted.
  2. [Section 4.1 and Appendix B] The hyperparameters are selected on the same benchmark used for the headline results. Figure 4 sweeps the mask ratio delta and the inter-modal alignment weight alpha on VQA v2 DI; Figure 9 sweeps the intra-recovery weight beta on the same setting; and Tables 9, 10, and 11 select prompt counts, recovery phases, and loss variants using VQA v2 DI. No separate validation split is described. This creates a risk of selection bias, because the reported gains may reflect tuned hyperparameters rather than a robust advantage. The authors should either use a held-out validation split for hyperparameter selection or show that the chosen values perform comparably across all reported settings.
  3. [Appendix B.1, Eqs. (12)-(13)] The modality-balance diagnostics used to support the mechanism claim are not validated and are not defined precisely enough to be reproduced. The paper does not specify how a vision-only or question-only input is constructed for VQA, which is nontrivial because the decoder presumably still requires both an image and a question to produce an answer. Moreover, the paper does not establish a causal link between improvement on these custom metrics and the accuracy gains: the ablation in Table 3 does not control for the extra loss terms or additional prompt capacity introduced by the recovery module. The authors should specify the input construction, validate the diagnostics on a known imbalance case, and include controlled ablations that separate the effect of the alignment losses from the effect of the extra parameters.
  4. [Section 2 and Section 4.2] The comparison omits at least two prior CVQA methods discussed in the paper. Section 2 cites Symbolic Replay [8] as a prompt-based CVQA method and also mentions the modality-aware feature distillation approach [5], but Section 4.2 compares only six general continual-learning methods and three CVQA methods ([4], [6], and [9]). If these cited methods are applicable to the same experimental protocol, their absence weakens the claim that MM-Prompt "surpasses prior approaches." Adding these baselines, or explicitly explaining why they are not comparable, would substantially strengthen the paper.
minor comments (6)
  1. [Throughout] The naming is inconsistent: "Cross-Modal Prompts Query" and "Cross-Modal Prompt Query" (also "Recovery" vs "Recoveries") are used interchangeably. Please standardize the terminology.
  2. [Section 3.3, Eq. (10)] The phrase "vice verse" should be "vice versa."
  3. [Section 4.4] The sentence "these results reveal the complementary neatural of our two components" contains a typo; it should presumably read "complementary nature."
  4. [Table 1] The table is extremely hard to read because large blocks of numbers are run together without clear column separation. The table would benefit from spacing, vertical rules, or separate sub-tables for VQA v2 and NExT-QA.
  5. [Figure 8] The vertical axis label "Acc After Normalized" is grammatically incomplete; it should be "Normalized Accuracy" or "Accuracy After Normalization."
  6. [References] Reference [7] appears to be a duplicate of Reference [1] (REVIVE) and should be removed or replaced.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: MM-Prompt is an empirical prompt-tuning method evaluated against external baselines on standard benchmarks, and no reported result reduces to its own inputs by construction.

full rationale

The paper's central claim is that MM-Prompt outperforms previous prompt-based methods in continual VQA. This claim is supported by accuracy and forgetting numbers measured on VQA v2 and NExT-QA against external baselines (Table 1), not by a derivation that assumes the conclusion. The auxiliary losses (L_intra, L_inter, L_qk-align in Eq. 11) are training objectives, not predictions, and the reported accuracies are independent evaluation outcomes. The modality-balance diagnostics in Appendix B.1 (Eqs. 12–13) are post-hoc measurements computed from model outputs; although they are custom metrics and their interpretation could be debated, they are not used to define or force the accuracy numbers, so they do not constitute a circular step. The self-citations in the related-work section (e.g., refs. [14–16, 36]) are not load-bearing for the main performance claim. The Limitations paragraph notes that the random masking strategy may lose important information; this is an acknowledged weakness but not a circular dependency. Concerns about single-run results, missing error bars, and hyperparameters selected on the same benchmark are experimental-validity issues rather than circularity. Accordingly, no circular step is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several hand-tuned hyperparameters and design postulates; no new physical or conceptual entities are introduced.

free parameters (5)
  • mask ratio δ = 0.05
    Tuned on VQA v2 DI (Fig 4a) as the best AP; controls the fraction of prompt dimensions masked before recovery.
  • inter-modal alignment loss weight α = 1.0
    Tuned on VQA v2 DI (Fig 4b); balances L_inter against the task loss.
  • intra-modal recovery loss weight β = 0.3
    Tuned on VQA v2 DI (Fig 9); chosen by peak AP.
  • prompt counts (QG, QV, EG, EV) = 40, 60, 80, 120
    Selected from Table 9 to maximize AP on VQA v2 DI; more capacity is given to visual and expert prompts.
  • top-k prompt retrieval count k = not reported
    Used in Eq. 2 for selecting top-k similar prompts; the paper does not state its value, yet it affects selection.
assumptions (5)
  • domain assumption The pretrained transformer backbone is frozen and only prompts are updated during continual learning.
    Adopted from prompt-based continual learning (DualPrompt, L2P); the paper does not question this design.
  • domain assumption The CVQA task decompositions (QI, CI, DI) and data splits follow VQACL and are realistic.
    The paper builds on this setting; if the splits contain leakage or are unrepresentative, the benchmark comparisons lose meaning.
  • domain assumption Language is the dominant modality in VQA and is a main cause of performance degradation over time.
    Cited from [18]; the entire motivation of modality imbalance depends on this.
  • ad hoc to paper Applying identical random masks to both modality prompt sets forces the model to perform genuine cross-modal integration during recovery.
    This is a design postulate of the paper; no independent evidence is given, and the limitation section acknowledges random masking may lose important information.
  • ad hoc to paper The alignment losses L_inter and L_intra prevent representational drift without hurting task learning.
    Evaluated only indirectly through final accuracy; no direct monitoring of drift is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering." pith.science (2026). https://pith.science/paper/TGK5BXU5

@misc{pith2026250519455,
  author       = {Pith},
  title        = {Pith review of: MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TGK5BXU5}},
  note         = {Machine review of arXiv:2505.19455}
}
read the original abstract

Continual Visual Question Answering (CVQA) based on pre-trained models(PTMs) has achieved promising progress by leveraging prompt tuning to enable continual multi-modal learning. However, most existing methods adopt cross-modal prompt isolation, constructing visual and textual prompts separately, which exacerbates modality imbalance and leads to degraded performance over time. To tackle this issue, we propose MM-Prompt, a novel framework incorporating cross-modal prompt query and cross-modal prompt recovery. The former enables balanced prompt selection by incorporating cross-modal signals during query formation, while the latter promotes joint prompt reconstruction through iterative cross-modal interactions, guided by an alignment loss to prevent representational drift. Extensive experiments show that MM-Prompt surpasses prior approaches in accuracy and knowledge retention, while maintaining balanced modality engagement throughout continual learning.

Figures

Figures reproduced from arXiv: 2505.19455 by the authors.

Figure 1
Figure 1. Comparison between traditional prompt-based CVQA approaches and MM-Prompt. Previous meth [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Detailed MM-Prompt Components for (a) Cross-Modal Prompts Query and (b) Cross-Modal Prompts [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison between existing prompt￾based approaches and our MM-Prompt model. 35.143 35.629 36.223 35.91 35.54 34.6 34.8 35 35.2 35.4 35.6 35.8 36 36.2 36.4 0.01 0.03 0.05 0.07 0.1 AP (%) (a) Effect of different δ 35.742 36.105 36.223 35.8 35.232 34.6 34.8 35 35.2 35.4 35.6 35.8 36 36.2 36.4 0.5 0.7 1 1.2 1.5 AP (%) (b) Effect of different α [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Bottom up attention visualization[45] during inference on Dual Prompt [10] and MM-Prompt. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visualization of injected prompts and input feature using t-SNE [46]. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Time spent processing 100 samples. 0 1 2 3 4 5 6 1 2 3 4 5 6 7 CW DL DB Acc After Normalized First Task: Task (a) Dual Prompt [10] 0 1 2 3 4 5 6 1 2 3 4 5 6 7 CW DL DB Acc After Normalized First Task: Task (b) MM-Prompt [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Comparison with different task orders in [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Effect of different Lintra ratio. target focus that yields “no”. These patterns persist across other tasks involving abstract reasoning and lighting assessment. These improvements arise from the design of MM-Prompt, where cross￾modal prompt query enriches queries with …
Figure 10
Figure 10. Figure 10: Bottom up attention visualization[45] during inference on Dual Prompt [10] and MM-Prompt. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Failure cases for MM-Prompt 20 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Group Preference Collapse in Personalized Multimodal Large Language Models

    cs.AI 2026-06 conditional novelty 6.0 of 10

    Modeling user preferences as shared prototypes plus personalized residuals — with imbalance-aware contrastive learning and hierarchical LoRA routing — substantially reduces drift toward majority answers in personalize...

Reference graph

Works this paper leans on

47 extracted references · 43 canonical work pages · cited by 1 Pith paper

  1. [8]

    Symbolic replay: Scene graph as prompt for continual learning on VQA task

    Stan Weixian Lei, Difei Gao, Jay Zhangjie Wu, Yuxuan Wang, Wei Liu, Mengmi Zhang, and Mike Zheng Shou. Symbolic replay: Scene graph as prompt for continual learning on VQA task. InProceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 139–149, 2023

    work page 2023

  2. [5]

    En- hancing continual learning in visual question answering with modality-aware feature distilla- tion.arXiv preprint arXiv:2406.19297, 2024

    Malvina Nikandrou, Georgios Pantazopoulos, Ioannis Konstas, and Alessandro Suglia. En- hancing continual learning in visual question answering with modality-aware feature distilla- tion.arXiv preprint arXiv:2406.19297, 2024

    arXiv 2024

  3. [4]

    Vqacl: A novel visual question answering continual learning setting

    Xi Zhang, Feifei Zhang, and Changsheng Xu. Vqacl: A novel visual question answering continual learning setting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19102–19112, 2023

    work page 2023

  4. [6]

    Decouple before interact: Multi-modal prompt learning for continual visual question answering

    Zi Qian, Xin Wang, Xuguang Duan, Pengda Qin, Yuhong Li, and Wenwu Zhu. Decouple before interact: Multi-modal prompt learning for continual visual question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2953–2962, 2023

    work page 2023

  5. [9]

    CluMo: Cluster-based Modality Fusion Prompt for Continual Learning in Visual Question Answering

    Yuliang Cai and Mohammad Rostami. CluMo: Cluster-based modality fusion prompt for continual learning in visual question answering.arXiv preprint arXiv:2408.11742, 2024

    work page Pith review arXiv 2024

  6. [2]

    VLC-BERT: Visual question answering with contextualized commonsense knowledge

    Sahithya Ravi, Aditya Chinchure, Leonid Sigal, Renjie Liao, and Vered Shwartz. VLC-BERT: Visual question answering with contextualized commonsense knowledge. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1155– 1165, 2023

    work page 2023

  7. [3]

    Align before fuse: Vision and language representation learning with momen- tum distillation

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momen- tum distillation. InAdvances in Neural Information Processing Systems (NeurIPS), pages 9694–9705, 2021

    work page 2021

  8. [7]

    RE- VIVE: Regional visual representation matters in knowledge-based visual question answering

    Yuanze Lin, Yujia Xie, Dongdong Chen, Yichong Xu, Chenguang Zhu, and Lu Yuan. RE- VIVE: Regional visual representation matters in knowledge-based visual question answering. InAdvances in Neural Information Processing Systems (NeurIPS), pages 10560–10571, 2022

    work page 2022

Show all 47 references
  1. [10]

    DualPrompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. DualPrompt: Complementary prompting for rehearsal-free continual learning. InEuropean Conference on Computer Vision (ECCV), pages 631–648. S...

    2022

  2. [11]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 139...

    2022

  3. [12]

    CODA-Prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning

    James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. CODA-Prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning. InProceed- ings of the ...

    2023

  4. [13]

    Semantic residual prompts for continual learning

    Martin Menabue, Emanuele Frascaroli, Matteo Boschini, Enver Sangineto, Lorenzo Bonicelli, Angelo Porrello, and Simone Calderara. Semantic residual prompts for continual learning. In European Conference on Computer Vision (ECCV), pages 1–18. Springer, 2024

    2024

  5. [14]

    Multi-domain multi- task rehearsal for lifelong learning

    Fan Lyu, Shuai Wang, Wei Feng, Zihan Ye, Fuyuan Hu, and Song Wang. Multi-domain multi- task rehearsal for lifelong learning. InProceedings of the AAAI Conference on Artificial Intel- ligence (AAAI), pages 8819–8827, 2021. 11

    2021

  6. [15]

    Exploring example influence in continual learning

    Qing Sun, Fan Lyu, Fanhua Shang, Wei Feng, and Liang Wan. Exploring example influence in continual learning. 2022

    2022

  7. [16]

    Measuring asymmetric gradient discrepancy in parallel continual learning

    Fan Lyu, Qing Sun, Fanhua Shang, Liang Wan, and Wei Feng. Measuring asymmetric gradient discrepancy in parallel continual learning. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11411–11420, 2023

    2023

  8. [17]

    MAPLE: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fa- had Shahbaz Khan. MAPLE: Multi-modal prompt learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19113–19122, 2023

    2023

  9. [18]

    Overcoming language priors in visual question answering with adversarial regularization

    Sainandan Ramakrishnan, Aishwarya Agrawal, and Stefan Lee. Overcoming language priors in visual question answering with adversarial regularization. InAdvances in Neural Informa- tion Processing Systems (NeurIPS), page 152–164, 2018

    2018

  10. [19]

    Making the v in VQA matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in VQA matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6904–6...

    2017

  11. [20]

    Lawrence Zitnick, and Devi Parikh

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. VQA: Visual question answering. InProceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2425–2433, 2015

    2015

  12. [21]

    A lifelong learning perspective for mobile robot control

    Sebastian Thrun. A lifelong learning perspective for mobile robot control. InProceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 201–214, 1995

    1995

  13. [22]

    Pre- venting zero-shot transfer degradation in continual learning of vision-language models

    Zangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin, Xiangyu Yue, and Yang You. Pre- venting zero-shot transfer degradation in continual learning of vision-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 19125–19136, 2023

    2023

  14. [23]

    Bakker, Nicu Sebe, and Michael S

    Mingrui Lao, Nan Pu, Yu Liu, Zhun Zhong, Erwin M. Bakker, Nicu Sebe, and Michael S. Lew. Multi-domain lifelong visual question answering via self-critical distillation. InProceedings of the 31st ACM International Conference on Multimedia (ACM MM), pages 4747–4758, 2023

    2023

  15. [24]

    Boosting continual learning of vision-language models via mixture-of-experts adapters

    Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, and You He. Boosting continual learning of vision-language models via mixture-of-experts adapters. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23219–23230, 2024

    2024

  16. [25]

    Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting

    Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. InProceed- ings of the International Conference on Machine Learning (ICML), pages 3925–3934. PMLR, 2019

    2019

  17. [26]

    Ex- perience replay for continual learning

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Ex- perience replay for continual learning. InAdvances in Neural Information Processing Systems (NeurIPS), pages 348–358, 2019

    2019

  18. [27]

    Dark experience for general continual learning: A strong, simple baseline

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: A strong, simple baseline. InAdvances in Neural Information Processing Systems (NeurIPS), pages 15920–15930, 2020

    2020

  19. [28]

    A continual learning survey: Defying forgetting in classification tasks.IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 3366–3385, 2021

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale ˇs Leonardis, Gre- gory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks.IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 3366–...

    2021

  20. [29]

    Yu, and Irwin King

    Dianzhi Yu, Xinni Zhang, Yankai Chen, Aiwei Liu, Yifei Zhang, Philip S. Yu, and Irwin King. Recent advances of multimodal continual learning: A comprehensive survey.arXiv preprint arXiv:2410.05352, 2024. 12

    2024

  21. [30]

    Balanced multimodal learning via on-the-fly gradient modulation

    Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu. Balanced multimodal learning via on-the-fly gradient modulation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8238–8247, 2022

    2022

  22. [31]

    Pre-trained models: Past, present and future.AI Open, pages 225–250, 2021

    Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, et al. Pre-trained models: Past, present and future.AI Open, pages 225–250, 2021

    2021

  23. [32]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, pages 1–67, 2020

    2020

  24. [33]

    LXMERT: Learning cross-modality encoder representations from transformers.arXiv preprint arXiv:1908.07490, 2019

    Hao Tan and Mohit Bansal. LXMERT: Learning cross-modality encoder representations from transformers.arXiv preprint arXiv:1908.07490, 2019

    1908 arXiv

  25. [34]

    DyTox: Trans- formers for continual learning with dynamic token expansion

    Arthur Douillard, Alexandre Ram ´e, Guillaume Couairon, and Matthieu Cord. DyTox: Trans- formers for continual learning with dynamic token expansion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9285–9295, 2022

    2022

  26. [35]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InProceedings of the International Confer- ence o...

    2021

  27. [36]

    Understanding driving risks via prompt learning

    Yubo Chang, Fan Lyu, Zhang Zhang, and Liang Wang. Understanding driving risks via prompt learning. InIEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 3637–3642, 2024

    2024

  28. [37]

    Attention bottlenecks for multimodal fusion

    Arsha Nagrani, Shan Yang, Anurag Arnab, Aren Jansen, Cordelia Schmid, and Chen Sun. Attention bottlenecks for multimodal fusion. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  29. [38]

    Difnet: Boosting visual information flow for image captioning

    Mingrui Wu, Xuying Zhang, Xiaoshuai Sun, Yiyi Zhou, Chao Chen, Jiaxin Gu, Xing Sun, and Rongrong Ji. Difnet: Boosting visual information flow for image captioning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2022

    2022

  30. [39]

    Aligning visual regions and textual concepts for semantic-grounded image representations

    Fenglin Liu, Yuanxin Liu, Xuancheng Ren, Xiaodong He, and Xu Sun. Aligning visual regions and textual concepts for semantic-grounded image representations. InAdvances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  31. [40]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll ´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009, 2022

    2022

  32. [41]

    A survey on masked autoencoder for self-supervised learning in vision and beyond

    Chaoning Zhang, Chenshuang Zhang, Junha Song, John Seon Keun Yi, Kang Zhang, and In So Kweon. A survey on masked autoencoder for self-supervised learning in vision and beyond. arXiv preprint arXiv:2208.00173, 2022

    2022 arXiv

  33. [42]

    Can we gain more from orthogonality regularizations in training deep networks? InAdvances in Neural Information Processing Systems (NeurIPS), 2018

    Nitin Bansal, Xiaohan Chen, and Zhangyang Wang. Can we gain more from orthogonality regularizations in training deep networks? InAdvances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  34. [43]

    NExT-QA: Next phase of question answering to explaining temporal actions

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. NExT-QA: Next phase of question answering to explaining temporal actions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9777–9786, 2021

    2021

  35. [44]

    A simple weight decay can improve generalization

    Anders Krogh and John Hertz. A simple weight decay can improve generalization. InAdvances in Neural Information Processing Systems (NeurIPS), pages 950–957, 1991

    1991

  36. [45]

    Bottom-up and top-down attention for image captioning and visual question answering

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. InProceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), ...

    2018

  37. [46]

    Visualizing data using t-SNE.Journal of Machine Learning Research, pages 2579–2605, 2008

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE.Journal of Machine Learning Research, pages 2579–2605, 2008

    2008

  38. [47]

    Scaling instruction-finetuned language models.Journal of Machine Learning Research, 2024

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models.Journal of Machine Learning Research, 2024

    2024

  39. [48]

    Plus”, “Mean Pooling

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. InProceedings of the Annual Mee...

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.