Pith. sign in

REVIEW 5 major objections 6 minor 3 cited by

Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering

T0 review · 5 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In continual visual question answering, storing only past question text—no images—and distilling attention from the previous model beats every tested method that stores full image-question pairs, reaching 39.25% average performance on…

desk verdict A solid, well-ablated paper on question-only replay for continual VQA that deserves review, but its central 'sufficiency' claim should be softened and it needs error bars. read the letter →

arxiv 2502.04469 v2 pith:XS6TBDEX submitted 2025-02-06 cs.CV cs.AI

Imad Eddine Marouf , Enzo Tartaglione , Stephane Lathuiliere , Joost van de Weijer This is my paper · ORCID
classification cs.CVcs.AI
keywords continuallearningvisualquestionansweringcatastrophicforgettingquestion-onlyreplayattentiondistillationout-of-answer-setproblemVQACLprivacy-preservingmachine
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to establish a counterintuitive result in continual visual question answering: to keep a model from forgetting how to answer past questions, you do not need to keep any of the past images. The proposed method, QUAD, stores only the text of past questions, replays them against current-task images, and uses two regularizers—soft pseudo-labels from the previous model and cross-entropy alignment of attention maps—to hold the model's knowledge in place. On the VQACL benchmark the method reaches 39.25% average performance on VQAv2 and 31.70% on NExT-QA, both above the strongest image-storing baseline (VQACL at 37.46% and 30.86%), with lower forgetting. If this holds, the practical payoff is that privacy-conscious continual VQA is not a compromise: storage falls from image-plus-question to question-text alone, and performance does not drop. The paper is candid that visually heavy skills such as fine-grained object type identification still suffer without stored images, so the claim is about sufficiency for the benchmark's skill mix rather than for all VQA skills.

What carries the argument

The load-bearing machinery is the stability loss $L_{\mathrm{Stability}} = L_{\mathrm{QR}} + L_{\mathrm{ACD}}$, evaluated on pairs $(x_t, q_m)$ of a current-task image and a stored past question. $L_{\mathrm{QR}}$ is a cross-entropy between the current model's output and the previous model's soft output on that pair---soft pseudo-labels, deliberately not argmaxed, so the full output distribution is preserved. $L_{\mathrm{ACD}}$ is a cross-entropy between the previous and current models' softmax-normalized self-attention maps, summed over all layers and heads, which weights corrections toward highly attended regions and leaves low-attended regions flexible. A third, selection-side mechanism carries the argument: questions are drawn from memory only if their object category matches the current subtask's object group, which keeps replayed pairs semantically coherent (a counting question is not paired with an image of cows when the current task counts cars). The named problem the mechanism targets is the out-of-answer-set problem, the multimodal analogue of class recency bias, where fine-tuning overfits the answer vocabulary of the newest task.

What would settle it

The decisive experiment is to break the object-category match between stored questions and current images: permute the five object groups per subtask so the images a question is paired with never contain the objects the question asks about, keeping the model, losses, and memory buffer otherwise identical. If QUAD's average performance on VQAv2 falls back toward the memory-free baseline rather than staying near 39%, the shared-category pairing is the load-bearing assumption; if it stays high, question replay generalizes beyond object overlap.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that question-only replay plus attention consistency distillation makes past images dispensable in the VQACL setting. The question-only replay term $L_{\mathrm{QR}}$ builds new image-question pairs from current images and stored past questions—matched so the question's object category and the image's content overlap—and trains the current model to imitate the previous model's soft output distribution on these pairs, without the argmax. The attention-consistency term $L_{\mathrm{ACD}}$ distills the previous model's normalized self-attention maps into the current model, preserving both intra-modal (text-text, image-image) and inter-modal (text-image) focus patterns. Together the two terms counteract the out-of-answer-set problem, the paper's name for the tendency of sequential fine-tuning to collapse all responses into the latest task's answer vocabulary. The empirical claim is that this recipe outperforms every tested method that does store images, on standard and on novel-composition tests, which the paper reads as evidence that questions alone can carry the visual-linguistic associations needed to mitigate forgetting.

Load-bearing premise

The method works only if pairing a stored past question with a current-task image that shares the same object category recreates enough of the original image-question association for the previous model's answers to be a trustworthy teaching signal.

Editorial extensions

If this is right

  • On the VQACL benchmark, a model that never stores images forgets less than every tested image-storing method: 4.91% average forgetting on VQAv2 versus 5.99% for the best prior method, and 2.91% versus 4.12% on NExT-QA.
  • Storage cost drops from image-plus-question to question-text alone, reducing the memory footprint from $O(N \cdot (I + L_q + L_a))$ to $O(N \cdot L_q)$ with less than 5% GPU-memory impact, since the number of processed pairs is unchanged.
  • Performance on novel compositions—unseen object-skill pairings—lands within 0.75 percentage points of standard performance on VQAv2, so the question-only signal transfers to combinations the model never trained on.
  • Larger memory buffers help monotonically, and object-matched question selection beats random pairing at every buffer size, so the selection criterion, not the buffer alone, is doing the work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same trick may transfer to other multimodal continual settings—video question answering, image captioning, visual dialogue—where the text side is the cheaper and less sensitive modality to retain; the paper does not test these settings.
  • The visible weakness on 'type' tasks suggests a hybrid that the paper only gestures at: keep a small buffer of images exclusively for visually intensive question types and rely on question replay everywhere else; per-task AP curves could be measured against how many images per type are retained.
  • Because question text can still correlate with sensitive visual content (a question about a person's identity is not anonymous), QUAD reduces storage-related privacy risk but does not eliminate information leakage; an adversarial re-identification test on stored questions would quantify what remains.
  • The 'judge' task jump (62.6% with the full method versus 35.0% with replay alone, per Fig. 4) is attributed to attention distillation; a targeted ablation that freezes $L_{\mathrm{ACD}}$ only for spatially demanding tasks would test whether attention alignment is the carrier or whether soft pseudo-labels alone suffice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper addresses continual visual question answering under a new question-only rehearsal setting, VQACL-QR, in which past images are not stored. The proposed method, QUAD, combines question-only replay (Eq. 4), which pairs stored past questions with current-task images using pseudo-labels from the previous model, with attention consistency distillation (Eq. 5), which aligns self-attention distributions across tasks. Experiments on VQAv2 and NExT-QA, plus experiments with BLIP-2 and LLaVA, report state-of-the-art average performance and lower forgetting compared with image-storing baselines such as VQACL. Ablations isolate the contributions of the two loss terms, and a memory-size sensitivity study is provided.

Significance. If the claims hold, the central result is significant: it would show that storing only questions can mitigate forgetting in continual VQA even when image-rehearsal methods store thousands of image-question pairs, with clear benefits for memory and privacy. The paper's strengths include a clearly specified problem setting, public code, component ablations in Table 3, comparison of attention-distillation variants in Table 4, and extensions to large pretrained vision-language models. However, the paper currently overstates the sufficiency claim relative to its own evidence: the novel-composition results on NExT-QA do not actually beat the best image-storing baseline, and the main comparisons lack error bars or statistical tests.

major comments (5)
  1. [§4.2, Table 1] The text in §4.2 states that QUAD achieves "top AP scores of 40.00% on VQAv2 and 33.85% on NExT-QA" for novel composition testing, but Table 1 reports QUAD at 33.21% for NExT-QA novel composition while VQACL reports 33.85%. Because the paper's headline claim is that question-only storage surpasses image-storing methods, this contradiction is load-bearing: on NExT-QA novel compositions, QUAD does not outperform VQACL. The text must be corrected and the claim tempered accordingly.
  2. [Tables 1–2, §4.2] All main results are reported as single runs without standard deviations or statistical tests. For example, QUAD versus VQACL on the standard test is 39.25% vs 37.46% on VQAv2 and 31.70% vs 30.86% on NExT-QA; the latter margin is 0.84 percentage points, which could easily be within run-to-run variation. The paper should report mean±std over at least three random seeds (or confidence intervals) for the main tables before claiming state-of-the-art superiority.
  3. [Abstract, §4.2, §8, Eq. (4)] The claim that "storing only questions is sufficient to mitigate forgetting" is broader than the evidence. Eq. (4) only provides meaningful replay when current-task images share object categories with stored questions, and the paper's own limitation section (§8) concedes that tasks requiring detailed visual or spatial reasoning, such as 'type' in Fig. 4, still degrade and that storing representative images "may be necessary" for high-fidelity visual reasoning. The paper should either restrict the sufficiency claim to object-recurrent, conceptually driven tasks or provide a quantitative task-level analysis showing how much forgetting is actually prevented.
  4. [Eq. (4), §3.3] The replay loss relies on soft pseudo-labels generated by the previous model θ_{t-1} for image-question pairs (x_t, q_m) that the teacher has never seen. The paper does not measure the reliability of these pseudo-labels, nor does it compare against an oracle-label variant. Since the entire question-only replay mechanism depends on these cross-pairs being semantically coherent and the pseudo-labels being trustworthy, the authors should report pseudo-label accuracy on the replayed pairs or ablate with ground-truth answers to establish that the replay signal is not systematically misleading.
  5. [§14, Fig. 2] The controlled out-of-answer-set experiment uses only 10 questions per task and no error bars, so the confusion matrices in Fig. 2 cannot quantitatively support the claimed severity of the phenomenon. The paper should report the full evaluation set with confidence intervals, or explicitly present Fig. 2 as an illustrative example rather than as an empirical evaluation.
minor comments (6)
  1. [Fig. 1] The caption mentions "Without replay: Blue" but does not explain what answer this refers to or which image-question pair produced it; please clarify.
  2. [§3.3] The example pairs counting cars with the stored question "What's the color of the car?", which is not a counting question; this obscures the intended object-matched selection. Clarify that selection matches object categories, not question types or reasoning skills.
  3. [Supp. §7 and Introduction] There are several typos, e.g., "distilaltion" in Supp. §7 and "lean new skills" in the Introduction; please proofread the text.
  4. [Table 1] The legend for the memory-type column is garbled ("? and /ctre"); the symbols used for question-only versus image-plus-question memory should be defined cleanly and consistently.
  5. [Eq. (5)] Cross-entropy is applied to two-dimensional self-attention maps; please specify how the maps are flattened or normalized so that LCE is well-defined as a distributional loss.
  6. [Supp. §12] The sensitivity analysis for λ is reported only on VQAv2; since λ is a central hyperparameter, the same analysis should be shown on NExT-QA or justified as unnecessary.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: QUAD's losses are defined independently of the target metrics and the central 'questions-only is sufficient' claim is an empirical comparison against external benchmarks.

full rationale

The paper's central claim (storing only past questions is sufficient to mitigate forgetting) is supported by AP and Forgetting scores on VQAv2 and NExT-QA computed against ground-truth answers and external baselines, not by construction from the loss definitions. LQR (Eq. 4) is a standard self-distillation replay term that pairs current images with stored questions and uses the previous model's soft outputs as targets; it is a training regularizer, not the evaluation metric. LACD (Eq. 5) aligns attention distributions and is ablated in Tables 3-4, with AP as the external criterion. The hyperparameter λ is fixed at 0.5 and its sensitivity is reported in Fig. 9, so the headline numbers are not fitted outcomes. The object-matched question selection in Sec. 3.3 is benchmarked against random pairing in Fig. 6 rather than assumed. Baseline comparisons use independently defined methods (ER, DER, VS, VQACL). The paper's own Sec. 8 concedes that visually detailed tasks such as 'type' may still require images, which is a limitation and a conditional framing of the sufficiency claim, not a circular reduction. Self-citations (e.g., [56], [63]) appear in related-work and baseline contexts and are not load-bearing for the central result. No step in the derivation reduces to its own input by definition.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The paper's central claim rests on a few domain assumptions about what information can be recovered from questions alone, plus standard hyperparameters (lambda, memory size). The method introduces no physical entities; the invented entries are a new evaluation setting and a new loss, both validated only within the paper.

free parameters (2)
  • lambda (stability weight) = 0.5
    Chosen via validation sweep (Fig. 9), not derived from first principles; it balances the plasticity and stability losses.
  • memory size = 5000 for VQAv2, 500 for NExT-QA
    Set to match prior baselines in the VQACL protocol, not optimized; the central comparison depends on this protocol choice.
assumptions (4)
  • domain assumption Questions paired with current images provide enough information to recover past visual-linguistic associations.
    Central to the replay mechanism in Sec. 3.3; conceded to fail for 'type' tasks in Sec. 8.
  • domain assumption Self-attention maps encode task-relevant knowledge and are the right representation to distill across tasks.
    Basis for ACD in Sec. 3.4; supported by ablations but not proven independently.
  • domain assumption Soft pseudo-labels from the previous model are reliable supervision for the current model.
    Standard distillation assumption, used in Eq. 4 for LQR.
  • domain assumption The VQACL task split (linguistic macro-tasks into visual subtasks) is a representative evaluation of continual VQA.
    Adopted from prior work [95] and used for all comparisons; the paper does not justify that this split reflects real-world continual VQA.
invented entities (2)
  • VQACL-QR setting
    purpose: A continual VQA protocol that permits storing only questions from past tasks, as a middle ground between memory-free and image-rehearsal settings.
    Introduced in this paper; no external benchmark or follow-up validation yet.
  • Attention Consistency Distillation (ACD) loss
    purpose: Regularizes attention distributions between current and previous models via cross-entropy on softmax-normalized attention maps.
    Novel objective; evidence is internal ablations (Tables 3 and 4) and attention-drift metrics in the supplementary.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering." pith.science (2026). https://pith.science/paper/XS6TBDEX

@misc{pith2026250204469,
  author       = {Pith},
  title        = {Pith review of: Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XS6TBDEX}},
  note         = {Machine review of arXiv:2502.04469}
}
read the original abstract

Continual Learning in Visual Question Answering (VQACL) requires models to acquire new visual-linguistic skills (plasticity) while preserving previously learned knowledge (stability). The inherent multimodality of VQACL exacerbates this challenge, as models must balance stability across visual and textual domains while adapting to novel objects and reasoning tasks. Existing methods, primarily designed for unimodal settings, often fall short in addressing this dual requirement. In this work, we present QUestion-only replay with Attention Distillation (QUAD), a novel approach for VQACL that leverages only past task questions for regularization. By eliminating the need to store visual data, QUAD not only reduces memory overhead, but also alleviates privacy concerns. Our method introduces a Question-only Replay mechanism that selectively reuses prior task questions to counteract overfitting to the answer space of the current task, addressing the problem out of answer set. Complementing this, we propose Attention Consistency Distillation to enforce both intra-modal and inter-modal attention consistency across tasks, preserving essential visual-linguistic associations. Extensive experiments on VQAv2 and NExT-QA demonstrate that QUAD significantly outperforms state-of-the-art methods, achieving robust performance in continual VQA. Code is available at: https://github.com/IemProg/QUAD.

Figures

Figures reproduced from arXiv: 2502.04469 by the authors.

Figure 1
Figure 1. Comparison of continual learning methods for Visual [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Out-of-Answer-Set Problem in Sequential Finetun￾ing. Confusion matrices compare Sequential Finetuning (left) and QUAD (right) across three sequentially trained tasks: Count￾ing, Action, and Color (y-axis: answers set vocabulary, x-axis: predicted answers). Diagonal values indicate model predictions on the current task, while off-diagonal shifts reveal predictions on previous tasks. Sequential Finetuning exhibits, mi… view at source ↗
Figure 3
Figure 3. Overview of QUAD. QUAD is composed of three components that jointly promote stability and plasticity in VQACL setting. (1) Question-Only Memory (M) stores questions from past tasks, without visual data. (2) Question-only replay (LQR) leverages answers generated by the previous model θ t−1 for new image-question pairs, encouraging the current model θ t to retain past knowledge. (3) Attention Consistency Distillation … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Plasticity/Stability analysis on VQAv2. Each matrix shows the performance of a model trained on tasks (rows) and evaluated on tasks sequentially (columns). The diagonal (highlighted in orange) represents in-domain performance, while off-diagonal elements in￾dicate cros…
Figure 6
Figure 6. Figure 6: highlights the advantage of object-matched ques￾tion selection compared to random pairing. We analyze 1000 2000 3000 4000 5000 Memory Size 37.8 38.1 38.4 38.7 39.0 39.3 AP (%) VQAv2 Random Obj-matched (QUAD) 100 200 300 400 500 Memory Size 28.0 28.8 29.6 30.4 31.2 AP (…
Figure 7
Figure 7. Figure 7: Entropy Difference. Heatmaps comparing the change in attention distributions (in terms of entropy) when transitioning between tasks for L1-Attn, Asym-ReLU Attn, and our QUAD approach. Warmer (red) cells indicate larger differences, while cooler (blue) cells indicate sm…
Figure 8
Figure 8. Figure 8: Cross-Attention Coherence. Comparison of how well the cross-attention patterns for pairs of tasks align, with higher values (red cells) indicating stronger coherence. By treating self-attention as a normalized probability distribution, our QUAD method main￾tains notabl…
Figure 9
Figure 9. Figure 9: Sensitivity to λ. The plot demonstrates the relationship between λ and average precision (AP) on VQAv2. Notably, QUAD consistently outperforms the standard VQACL baseline for λ ≥ 0.4, underscoring the effective￾ness of our tailored stability components—question-only re…
Figure 10
Figure 10. Figure 10: Comparison of feature distillation methods on [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting

    cs.CV 2025-08 unverdicted novelty 7.0 of 10

    The paper offers a comprehensive survey and proposes a new taxonomy for continual learning strategies in VLMs and MLLMs to combat catastrophic forgetting beyond traditional methods.

  2. Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Aligning images to multi-view caption cores while suppressing orthogonal residual text and disagreement-aware temperature improves robust zero-shot recognition and LVLM transfer.

  3. Continual Learning for Generative AI: From LLMs to MLLMs and Beyond

    cs.LG 2025-06 conditional novelty 4.0 of 10

    A survey that categorizes continual learning methods for generative models into architecture-based, regularization-based, and replay-based paradigms across four model families.

Reference graph

Works this paper leans on

109 extracted references · 63 canonical work pages · cited by 3 Pith papers

  1. [1]

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, Soham Ghosh, Am ´elie H ´eliou, Paul Jacob, Albert Q. Jiang, Kartik Khandelwal, Timoth ´ee Lacroix, Guillaume Lample, Diego Las Casas, Thibaut Lavril, Teven Le Scao, Andy Lo, William Mar...

    2024

  2. [2]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018. 2, 6, 7

    2018

  3. [3]

    Looking at words and points with attention: a benchmark for text-to-shape coherence

    Andrea Amaduzzi, Giuseppe Lisanti, Samuele Salti, and Luigi Di Stefano. Looking at words and points with at- tention: a benchmark for text-to-shape coherence. arXiv preprint arXiv:2309.07917, 2023. 3

    work page Pith review arXiv 2023

  4. [4]

    Lawrence Zitnick, and Devi Parikh

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In ICCV, 2015. 3

    2015

  5. [5]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In ICCV, 2015. 2

    2015

  6. [6]

    Riemannian walk for incremen- tal learning: Understanding forgetting and intransigence

    Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajan- than, and Philip HS Torr. Riemannian walk for incremen- tal learning: Understanding forgetting and intransigence. In ECCV, 2018. 6

    2018

  7. [7]

    Efficient lifelong learning with a- gem

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a- gem. arXiv preprint arXiv:1812.00420, 2018. 3

    arXiv 2018

  8. [8]

    Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Kumar Dokania, Philip H. S. Torr, and Marc’Aurelio Ranzato. Continual learning with tiny episodic memories. CoRR, abs/1902.10486, 2019. 2, 3, 6, 7

    arXiv 1902

Show all 109 references
  1. [9]

    Dokania, Philip H

    Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K. Dokania, Philip H. S. Torr, and Marc’Aurelio Ranzato. On tiny episodic memories in continual learning, 2019. 2, 3, 6, 7

    2019

  2. [10]

    Vindlu: A recipe for effective video-and-language pretraining

    Feng Cheng, Xizi Wang, Jie Lei, David Crandall, Mohit Bansal, and Gedas Bertasius. Vindlu: A recipe for effective video-and-language pretraining. In CVPR, 2023. 1, 2

    2023

  3. [11]

    Unifying vision-and-language tasks via text generation

    Jaemin Cho, Jie Lei, Hao Tan, and Mohit Bansal. Unifying vision-and-language tasks via text generation. 2021. 2

    2021

  4. [12]

    One vlm to keep it learning: Gen- eration and balancing for data-free continual visual question answering

    Deepayan Das, Davide Talon, Massimiliano Mancini, Yim- ing Wang, and Elisa Ricci. One vlm to keep it learning: Gen- eration and balancing for data-free continual visual question answering. IEEE, 2025. 4

    2025

  5. [13]

    Ratt: Recurrent attention to transient tasks for continual image captioning

    Riccardo Del Chiaro, Bartł omiej Twardowski, Andrew Bag- danov, and Joost van de Weijer. Ratt: Recurrent attention to transient tasks for continual image captioning. In NeurIPS,

  6. [14]

    Learning without mem- orizing

    Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. Learning without mem- orizing. In CVPR, 2019. 3, 4, 5, 1, 2

    2019

  7. [15]

    Sok: Model inversion attack land- scape: Taxonomy, challenges, and future roadmap

    Sayanton V Dibbo. Sok: Model inversion attack land- scape: Taxonomy, challenges, and future roadmap. In 2023 IEEE 36th Computer Security Foundations Symposium (CSF). IEEE, 2023. 1

    2023

  8. [16]

    Podnet: Pooled outputs distil- lation for small-tasks incremental learning

    Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distil- lation for small-tasks incremental learning. In ECCV, 2020. 3

    2020

  9. [17]

    Dytox: Transformers for continual learn- ing with dynamic token expansion

    Arthur Douillard, Alexandre Ram ´e, Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learn- ing with dynamic token expansion. In CVPR, 2022. 3

    2022

  10. [18]

    Memory efficient contin- ual learning with transformers

    Beyza Ermis, Giovanni Zappella, Martin Wistuba, Aditya Rawal, and Cedric Archambeau. Memory efficient contin- ual learning with transformers. NeurIPS, 2022. 3

    2022

  11. [19]

    Vision language transformers: A survey

    Clayton Fields and Casey Kennington. Vision language transformers: A survey. arXiv preprint arXiv:2307.03254 ,

    arXiv

  12. [20]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412,

    2010 arXiv

  13. [21]

    A unified continual learn- ing framework with general parameter-efficient tuning.arXiv preprint arXiv:2303.10070, 2023

    Qiankun Gao, Chen Zhao, Yifan Sun, Teng Xi, Gang Zhang, Bernard Ghanem, and Jian Zhang. A unified continual learn- ing framework with general parameter-efficient tuning.arXiv preprint arXiv:2303.10070, 2023. 3

    2023 arXiv

  14. [22]

    General data protection regulation

    General Data Protection Regulation GDPR. General data protection regulation. Regulation (EU) 2016/679 of the Eu- ropean Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the process- ing of personal data and on the free movement...

    2016

  15. [23]

    Exploring the frontier of vision- language models: A survey of current methodologies and future directions, 2024

    Akash Ghosh, Arkadeep Acharya, Sriparna Saha, Vinija Jain, and Aman Chadha. Exploring the frontier of vision- language models: A survey of current methodologies and future directions, 2024. 3

    2024

  16. [24]

    Anisotropy is inherent to self-attention in transformers

    Nathan Godey, ´Eric de la Clergerie, and Beno ˆıt Sagot. Anisotropy is inherent to self-attention in transformers. arXiv preprint arXiv:2401.12143, 2024. 5

    2024 arXiv

  17. [25]

    Deep Learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning . MIT Press, 2016. http : / / www . deeplearningbook.org. 2

    2016

  18. [26]

    Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning

    Dipam Goswami, Yuyang Liu, Bartł omiej Twardowski, and Joost van de Weijer. Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. In NeurIPS, 2023. 1

    2023

  19. [27]

    Psycholinguistics meets continual learning: Measuring catastrophic forgetting in visual question answer- ing

    Claudio Greco, Barbara Plank, Raquel Fern ´andez, and Raf- faella Bernardi. Psycholinguistics meets continual learning: Measuring catastrophic forgetting in visual question answer- ing. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistic...

    2019

  20. [28]

    Psycholinguistics meets continual learning: Measuring catastrophic forgetting in visual question answer- ing

    Claudio Greco, Barbara Plank, Raquel Fern ´andez, and Raf- faella Bernardi. Psycholinguistics meets continual learning: Measuring catastrophic forgetting in visual question answer- ing. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistic...

    2019

  21. [29]

    Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In CVPR, 2018

    Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In CVPR, 2018. 5

    2018

  22. [30]

    Distill- ing the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distill- ing the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 4

    2015 arXiv

  23. [31]

    Learning a unified classifier incrementally via rebalancing

    Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, 2019. 3

    2019

  24. [32]

    Cl-moe: Enhancing multi- modal large language model with dual momentum mixture- of-experts for continual visual question answering

    Tianyu Huai, Jie Zhou, Xingjiao Wu, Qin Chen, Qingchun Bai, Ze Zhou, and Liang He. Cl-moe: Enhancing multi- modal large language model with dual momentum mixture- of-experts for continual visual question answering. InCVPR,

  25. [33]

    Clevr: A diagnostic dataset for compositional language and elementary visual reasoning

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR, 2017. 2

    2017

  26. [34]

    Class- Incremental Learning by Knowledge Distillation with Adap- tive Feature Consolidation

    Minsoo Kang, Jaeyoo Park, and Bohyung Han. Class- Incremental Learning by Knowledge Distillation with Adap- tive Feature Consolidation. In CVPR, 2022. 3, 5

    2022

  27. [35]

    Measuring compositional generalization: A comprehensive method on realistic data

    Daniel Keysers, Nathanael Sch ¨arli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, et al. Measuring compositional generalization: A comprehensive method on realistic data. arXiv preprint arXiv:...

    1912 arXiv

  28. [36]

    Adam: A method for stochastic optimization

    Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015. 5

    2015

  29. [37]

    Overcoming catastrophic forgetting in neu- ral networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sc...

    2017

  30. [38]

    Overcoming catastrophic forgetting in neu- ral networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sc...

    2017

  31. [39]

    Deep weight factorization: Sparse learning through the lens of artificial symmetries

    Chris Kolb, Tobias Weber, Bernd Bischl, and David R¨ugamer. Deep weight factorization: Sparse learning through the lens of artificial symmetries. ICLR, 2025. 2

    2025

  32. [40]

    Shamma, Michael S

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. ...

    2017

  33. [41]

    Mitigating the diminishing effect of elastic weight consolidation

    Canasai Kruengkrai and Junichi Yamagishi. Mitigating the diminishing effect of elastic weight consolidation. In Pro- ceedings of the 29th International Conference on Compu- tational Linguistics, pages 4568–4574, Gyeongju, Republic of Korea, 2022. International Committee on Com...

    2022

  34. [42]

    Building machines that learn and think like people

    Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines that learn and think like people. Behavioral and brain sciences , 40:e253,

  35. [43]

    Less is more: Clipbert for video-and-language learning via sparse sampling

    Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L Berg, Mohit Bansal, and Jingjing Liu. Less is more: Clipbert for video-and-language learning via sparse sampling. In CVPR,

  36. [44]

    BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML,

  37. [45]

    How to configure good in-context sequence for visual question answering

    Li Li, Jiawei Peng, Huiyi Chen, Chongyang Gao, and Xu Yang. How to configure good in-context sequence for visual question answering. In CVPR, 2024. 2

    2024

  38. [46]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelli- gence, 40(12):2935–2947, 2017. 3

    2017

  39. [47]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 40(12):2935–2947, 2018. 3, 4

    2018

  40. [48]

    Belongie, Lubomir D

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll’a r, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. CoRR, abs/1405.0312,

    arXiv

  41. [49]

    Pri- vacy intelligence: A survey on image privacy in online social networks

    Chi Liu, Tianqing Zhu, Jun Zhang, and Wanlei Zhou. Pri- vacy intelligence: A survey on image privacy in online social networks. ACM Computing Surveys, 2022. 2

    2022

  42. [50]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 2

    2023

  43. [51]

    Pre-train, prompt, and predict: A systematic survey of prompting methods in nat- ural language processing

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hi- roaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in nat- ural language processing. ACM Computing Surveys, 55(9): 1–35, 2023. 3

    2023

  44. [52]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. NeurIPS, 2017. 3

    2017

  45. [53]

    Gradient episodic memory for continual learning, 2022

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning, 2022. 6

    2022

  46. [54]

    Augmented ge- ometric distillation for data-free incremental person reid

    Yichen Lu, Mei Wang, and Weihong Deng. Augmented ge- ometric distillation for data-free incremental person reid. In CVPR, 2022. 3

    2022

  47. [55]

    Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learn- ing

    Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learn- ing. In CVPR, 2021. 4, 5

    2021

  48. [56]

    Weighted ensemble models are strong continual learners

    Imad Eddine Marouf, Subhankar Roy, Enzo Tartaglione, and St´ephane Lathuili`ere. Weighted ensemble models are strong continual learners. In ECCV, 2024. 1

    2024

  49. [57]

    Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory

    James L McClelland, Bruce L McNaughton, and Randall C O’Reilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995. 1, 3

    1995

  50. [58]

    Catastrophic inter- ference in connectionist networks: The sequential learning problem

    Michael McCloskey and Neal J Cohen. Catastrophic inter- ference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. 1989. 1

    1989

  51. [59]

    An empirical investigation of the role of pre-training in lifelong learning

    Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role of pre-training in lifelong learning. arXiv preprint arXiv:2112.09153, 2021. 3

    2021 arXiv

  52. [60]

    Context- vqa: Towards context-aware and purposeful visual question answering

    Nandita Naik, Christopher Potts, and Elisa Kreiss. Context- vqa: Towards context-aware and purposeful visual question answering. In ICCV, 2023. 2

    2023

  53. [61]

    Task formulation matters when learning continually: A case study in visual question answer- ing

    Mavina Nikandrou, Lu Yu, Alessandro Suglia, Ioannis Kon- stas, and Verena Rieser. Task formulation matters when learning continually: A case study in visual question answer- ing. arXiv preprint arXiv:2210.00044, 2022. 2, 3, 5

    2022 arXiv

  54. [62]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Rai- son, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, L...

    2019

  55. [63]

    Towards exemplar-free continual learning in vision transformers: an account of at- tention, functional and weight regularization

    Francesco Pelosin, Saurav Jha, Andrea Torsello, Bogdan Ra- ducanu, and Joost van de Weijer. Towards exemplar-free continual learning in vision transformers: an account of at- tention, functional and weight regularization. In CVPR,

  56. [64]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. 2020. 5

    2020

  57. [65]

    Vlc-bert: Visual question answering with contextualized commonsense knowledge

    Sahithya Ravi, Aditya Chinchure, Leonid Sigal, Renjie Liao, and Vered Shwartz. Vlc-bert: Visual question answering with contextualized commonsense knowledge. 2023. 3

    2023

  58. [66]

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental clas- sifier and representation learning. In CVPR, 2017. 3

    2017

  59. [67]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIPS, 2015. 5

    2015

  60. [68]

    Task-recency bias strikes back: Adapting covariances in exemplar-free class incremental learning

    Grzegorz Rype ´s´c, Sebastian Cygert, Tomasz Trzcinski, and Bartłomiej Twardowski. Task-recency bias strikes back: Adapting covariances in exemplar-free class incremental learning. NeurIPS, 2025. 4, 5

    2025

  61. [69]

    Gradient pro- jection memory for continual learning

    Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient pro- jection memory for continual learning. In ICLR, 2021. 3

    2021

  62. [70]

    A-okvqa: A benchmark for visual question answering using world knowl- edge

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A benchmark for visual question answering using world knowl- edge. In ECCV, 2022. 1

    2022

  63. [71]

    Non-autoregressive sequence-to-sequence vision-language models

    Kunyu Shi, Qi Dong, Luis Goncalves, Zhuowen Tu, and Stefano Soatto. Non-autoregressive sequence-to-sequence vision-language models. In CVPR, 2024. 2

    2024

  64. [72]

    Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning

    James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning. In CVPR, 2023. 3

    2023

  65. [73]

    Climb: A continual learning benchmark for vision-and- language tasks

    Tejas Srinivasan, Ting-Yun Chang, Leticia Pinto Alva, Geor- gios Chochlakis, Mohammad Rostami, and Jesse Thoma- son. Climb: A continual learning benchmark for vision-and- language tasks. NeurIPS, 2022. 3

    2022

  66. [74]

    A privacy-preserving image retrieval scheme with access control based on searchable encryption in media cloud

    Miao Tian, Yushu Zhang, Yongming Zhang, Xiangli Xiao, and Wenying Wen. A privacy-preserving image retrieval scheme with access control based on searchable encryption in media cloud. Cybersecurity, 2024. 2

    2024

  67. [75]

    Analyzing multi-head self-attention: Spe- cialized heads do the heavy lifting, the rest can be pruned

    Elena V oita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Spe- cialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418, 2019. 5

    1905 arXiv

  68. [76]

    Learning soft labels via meta learning, 2021

    Nidhi Vyas, Shreyas Saxena, and Thomas V oice. Learning soft labels via meta learning, 2021. 4

    2021

  69. [77]

    Timmy S. T. Wan, Jun-Cheng Chen, Tzer-Yi Wu, and Chu- Song Chen. Continual learning for visual search with back- ward consistent feature embedding. In CVPR, 2022. 2, 3, 6, 7, 8

    2022

  70. [78]

    S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning

    Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. NeurIPS, 2022. 3

    2022

  71. [79]

    Learning adaptive axis attentions in fine-tuning: Be- yond fixed sparse attention patterns

    Zihan Wang, Jiuxiang Gu, Jason Kuen, Handong Zhao, Vlad Morariu, Ruiyi Zhang, Ani Nenkova, Tong Sun, and Jingbo Shang. Learning adaptive axis attentions in fine-tuning: Be- yond fixed sparse attention patterns. In Findings of the As- sociation for Computational Linguistics: AC...

    2022

  72. [80]

    Continual learning with lifelong vision trans- former

    Zhen Wang, Liu Liu, Yiqun Duan, Yajing Kong, and Dacheng Tao. Continual learning with lifelong vision trans- former. In CVPR, 2022. 1

    2022

  73. [81]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vin- cent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In ECCV,

  74. [82]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jen- nifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, 2022. 3

    2022

  75. [83]

    Unified coarse-to-fine alignment for video-text retrieval

    Ziyang Wang, Yi-Lin Sung, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Unified coarse-to-fine alignment for video-text retrieval. In ICCV, 2023. 1

    2023

  76. [84]

    Separating skills and concepts for novel visual question answering

    Spencer Whitehead, Hui Wu, Heng Ji, Rogerio Feris, and Kate Saenko. Separating skills and concepts for novel visual question answering. In CVPR, 2021. 3

    2021

  77. [85]

    Next-qa: Next phase of question-answering to explaining temporal actions

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question-answering to explaining temporal actions. In CVPR, pages 9777–9786, 2021. 6

    2021

  78. [86]

    Video graph transformer for video question answering

    Junbin Xiao, Pan Zhou, Tat-Seng Chua, and Shuicheng Yan. Video graph transformer for video question answering. In ECCV, 2022. 2

    2022

  79. [87]

    Multi-task learning with knowledge distillation for dense prediction

    Yangyang Xu, Yibo Yang, and Lefei Zhang. Multi-task learning with knowledge distillation for dense prediction. In ICCV, 2023. 5

    2023

  80. [88]

    Zero-shot video question answering via frozen bidirectional language models

    Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. Zero-shot video question answering via frozen bidirectional language models. NeurIPS, 2022. 1, 2

    2022

  81. [89]

    Data augmented flatness-aware gradient projection for continual learning

    Enneng Yang, Li Shen, Zhenyi Wang, Shiwei Liu, Guibing Guo, and Xingwei Wang. Data augmented flatness-aware gradient projection for continual learning. In ICCV, 2023. 3

    2023

  82. [90]

    Self-chained image-language model for video localization and question answering

    Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. Self-chained image-language model for video localization and question answering. arXiv preprint arXiv:2305.06988,

    arXiv

  83. [91]

    Suzanne Barber

    Razieh Nokhbeh Zaeem and K. Suzanne Barber. The effect of the gdpr on privacy policies: Recent progress and future promise. ACM Trans. Manage. Inf. Syst., 2020. 2

    2020

  84. [92]

    Privacy threats and protection in machine learning

    Jiliang Zhang, Chen Li, Jing Ye, and Gang Qu. Privacy threats and protection in machine learning. In Proceedings of the 2020 on Great Lakes Symposium on VLSI, 2020. 1

    2020

  85. [93]

    Overcoming generic knowledge loss with selec- tive parameter update

    Wenxuan Zhang, Paul Janson, Rahaf Aljundi, and Mohamed Elhoseiny. Overcoming generic knowledge loss with selec- tive parameter update. In CVPR, 2024. 2

    2024

  86. [94]

    Refin- ing pseudo labels with clustering consensus over generations for unsupervised object re-identification

    Xiao Zhang, Yixiao Ge, Yu Qiao, and Hongsheng Li. Refin- ing pseudo labels with clustering consensus over generations for unsupervised object re-identification. In CVPR, 2021. 4

    2021

  87. [95]

    Vqacl: A novel visual question answering continual learning setting

    Xi Zhang, Feifei Zhang, and Changsheng Xu. Vqacl: A novel visual question answering continual learning setting. In CVPR, 2023. 1, 2, 3, 5, 6, 7, 8

    2023

  88. [96]

    Continual sequence generation with adaptive compositional modules

    Yanzhe Zhang, Xuezhi Wang, and Diyi Yang. Continual sequence generation with adaptive compositional modules. In Proceedings of the 60th Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Pa- pers), pages 3653–3667, Dublin, Ireland, 2022. Associat...

    2022

  89. [97]

    Expandable subspace ensemble for pre-trained model- based class-incremental learning

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model- based class-incremental learning. In CVPR, 2024. 1 Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering Supplementary Material In...

    2024

  90. [98]

    We do not foresee any negative soci- etal impact from this work, as it does not involve the gen- eration of harmful or biased data

    Ethics Statement Our method, QUAD, is designed to improve continual learning in Visual Question Answering (VQACL) while maintaining generalization and privacy through distilaltion using questions-only. We do not foresee any negative soci- etal impact from this work, as it does...

  91. [99]

    Limitations of QUAD While QUAD effectively reduces storage requirements and enhances privacy by eliminating the need to store images, it may be suboptimal for tasks that heavily rely on detailed vi- sual or spatial reasoning. Certain VQA tasks, such as object classification, f...

  92. [100]

    Discussion about Attention Consistency Dis- tillation Problem setup. Consider a self-attention mechanism where the attention matrix at layer l, head k, for an input sequence x at task t is given by: At l,k(x) = QlK T l√ d , (6) where Ql, Kl ∈ RN ×d are the query and key matric...

  93. [101]

    Analysis of Attention Drift To assess the effectiveness of QUAD in mitigating at- tention drift in continual VQA, we compare it to L1- Attention Regularization (L1-Attn) [14] and Asymmetric ReLU-Attention Regularization (Asym-ReLU Attn) [63] using two metrics: Cross-Attention ...

  94. [102]

    This section analyzes storage requirements, computational com- plexity, and GPU memory usage of our text-only replay ap- proach compared to image-based methods

    Computational Analysis Efficient memory and storage management is crucial for continual VQA, where scalability is a key challenge. This section analyzes storage requirements, computational com- plexity, and GPU memory usage of our text-only replay ap- proach compared to image-...

  95. [103]

    9, which governs the trade-off between adaptation to new tasks (plasticity) and retention of prior knowledge (stability) in our Q UAD framework

    Effect of λ We investigate the sensitivity of our model to the balancing coefficient λ in Fig. 9, which governs the trade-off between adaptation to new tasks (plasticity) and retention of prior knowledge (stability) in our Q UAD framework. The results demonstrate that performa...

  96. [104]

    Pre-trained models/VQA architectures We extend our evaluation to recent continual learn- ing approaches—CL-MoE [32] and GaB [12]—using pretrained vision-language models BLIP-2 and LLaV A (Tabs. 5, 6). On BLIP-2, QUAD achieves the highest av- erage precision (AP = 50.27) and lo...

  97. [105]

    Red” instead of “Two

    Out-of-Answer-Set Problem Evaluation To empirically analyze the out-of-answer-set problem, we designed a controlled continual learning experiment within the VQACL setting. Our objective was to demonstrate how sequential fine-tuning without appropriate regulariza- tion leads to...

  98. [106]

    Detailed Description of the VQACL Setting This section provides a detailed overview of the Visual Question Answering Continual Learning (VQACL) setting, as introduced by [95]. The VQACL setting is designed to test a model’s ability to generalise and retain knowledge across a s...

  99. [107]

    Each dataset has been carefully structured into different tasks, which are used to evaluate the performance of our continual learning models

    Details of Evaluation Datasets In this section, we provide a detailed overview of the two datasets used in our evaluation: VQA v2 and NExT-QA. Each dataset has been carefully structured into different tasks, which are used to evaluate the performance of our continual learning ...

  100. [108]

    The sequential finetuning baseline (left) demon- strates severe forgetting, with consistently low off-diagonal values

    Extended Analysis of Plasticity/Stability Trade-Off Fig.10 compares the impact of three continual learning strategies on performance across tasks in the NExT-QA dataset. The sequential finetuning baseline (left) demon- strates severe forgetting, with consistently low off-diago...

  101. [109]

    To ensure a consistent evaluation, all methods are implemented using their official codebases and integrated into the same trans- former backbone as described in Section 5.1

    Continual Learning Methods We assess and benchmark five prominent continual learn- ing methods, encompassing two regularization techniques (EWC [38], MAS [2]) and three rehearsal-based methods (ER [9], DER [8], VS [77], and VQACL[95]). To ensure a consistent evaluation, all me...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.