Pith. sign in

REVIEW 5 cited by

Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.09181 v3 pith:NSRQRM3C submitted 2024-01-17 cs.LG

Junhao Zheng , Qianli Ma , Zhen Liu , Binquan Wu , Huawen Feng This is my paper
classification cs.LG
keywords forwardtransfertaskstuningcatastrophicforgettinginstructionmcit
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal Continual Instruction Tuning (MCIT) enables Multimodal Large Language Models (MLLMs) to meet continuously emerging requirements without expensive retraining. MCIT faces two major obstacles: catastrophic forgetting (where old knowledge is forgotten) and negative forward transfer (where the performance of future tasks is degraded). Although existing methods have greatly alleviated catastrophic forgetting, they still suffer from negative forward transfer. We discover a large discrepancy in different input embeddings by performing singular value decomposition (SVD) on input embeddings. This discrepancy results in the model learning irrelevant information for old and pre-trained tasks, leading to catastrophic forgetting and negative forward transfer. To address these issues, we propose Prompt Tuning with Positive Forward Transfer (Fwd-Prompt), a prompt-based method that projects the prompt gradient to the residual space to minimize interference between tasks and to the pre-trained subspace for reusing pre-trained knowledge. Our experiments demonstrate that Fwd-Prompt achieves state-of-the-art performance while updating fewer parameters and requiring no old samples. Our research illuminates the potential of continuously adapting MLLMs to new tasks under the instruction tuning paradigm and encourages future studies to explore MCIT.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Domain-Aware Scaling Laws Uncover Data Synergy

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Domain-aware scaling laws with fitted γ and σ synergy terms recover stable code-math interactions from observational LLM mixtures and correctly predict mixture rankings in controlled small-scale trainings.

  2. Modality-Inconsistent Continual Learning of Multimodal Large Language Models

    cs.LG 2024-12 unverdicted novelty 7.0 of 10

    The paper introduces the MICL scenario for MLLMs with modality and task shifts and proposes MoInCL using pseudo-target generation and instruction-based distillation, reporting gains over continual learning baselines o...

  3. When the Same Musical Knowledge Forgets Differently: A Clean Probe of Pathway-Dependent Forgetting

    cs.SD 2026-06 unverdicted novelty 6.0 of 10

    Acquisition route affects forgetting rates in multimodal models, with text-pathway knowledge forgetting faster than audio-pathway knowledge in music understanding tasks.

  4. Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMs

    cs.CV 2025-11 conditional novelty 6.0 of 10

    HPA is a post-training parameter-selection method that keeps safety-aligned multimodal LLMs safe and reduces forgetting during continual visual instruction tuning.

  5. Continual Gradient Low-Rank Projection Fine-Tuning for LLMs

    cs.LG 2025-07 conditional novelty 5.0 of 10

    GORP jointly trains LoRA and full-rank parameters inside a low-rank gradient subspace built from Adam first moments, reporting higher average accuracy and lower forgetting than O-LoRA and N-LoRA on LLM continual learn...

Pith tools