{"id":"e1e51c30-2693-45ef-9df4-4053221951b6","arxiv_id":"2501.09352","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"PAL combines modality-specific prompting with recursive least squares to achieve exemplar-free multi-modal class-incremental learning under missing-modality conditions.","lead":"This paper proposes PAL, a method that lets vision-language models learn new classes over time even when images or texts are missing from training data. It combines learned prompts with an analytical least-squares classifier to reduce forgetting without storing old examples.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RLS exact-equivalence theorem assumes a fixed feature extractor, but PAL retrains prompts each step, so Theorem 1 does not strictly apply to the reported method.","rationale":"The reader identified the same load-bearing concern that I find: the RLS recursion in Theorem 1 is derived under a fixed feature mapping, but PAL's prompt pools are updated via backpropagation at every incremental step, so the H matrices entering Eq. (13) are produced by different prompts. This is not a flaw in the algebraic proof of the theorem, which is correct for a truly fixed H; it is a mismatch between the theorem's assumptions and the algorithm's actual use. The empirical results may still be valid as an engineering contribution, and the ablations give useful evidence for the prompt and AL modules, but the paper's specific claim of exact equivalence to joint training and the 'step-invariant property' are overstated without a drift analysis or correction. This does not warrant rejection, but it does warrant a conditional verdict requiring either a demonstration that prompt drift is negligible or a revised theoretical statement. Other issues, such as the novelty claim relative to RebQ and the missing variance reporting, are secondary and do not affect the core correctness argument as directly. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":17649,"tokens_out":4008,"duration_ms":77105,"concrete_test":"Using the UPMC-Food101 70% missing-both protocol, cache the prompt state after each task and also retain a small validation subset of each task's data. After the final task, compute the feature drift for previous tasks: for each i, extract H_i^old with the prompts cached at task i and H_i^final with the final prompts; report ||H_i^old - H_i^final||_F / ||H_i^old||_F. Then compare the classifier W obtained by the incremental RLS recursion with the closed-form solution of Eq. (11) using all data and the final prompts. If the relative weight error or the accuracy gap on the held-out subsets exceeds a small threshold (e.g., 1% average accuracy), the fixed-feature assumption is violated and Theorem 1 cannot be invoked as the mechanism for PAL; if the gap is within run-to-run variation, the drift is negligible and the concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim is that the RLS recursion (Eqs. 13-14) yields exactly the same classifier as joint training on all seen data (Theorem 1 and its proof). This equivalence is only valid if every H_i in Eq. (11) is produced by the same feature mapping. In PAL, however, the modality-specific prompts P_v and P_t are updated by backpropagation at every task (Section III.D.1, Algorithm 1 step 4), and the features H_k used at step k are extracted from the newly tuned prompts. The stored H_1,...,H_{k-1} from earlier tasks were produced by older prompt states, so the column blocks in H_{1:k} are incompatible: they come from different functions. Eq. (13) is then not the minimizer of the joint least-squares objective under the final feature mapping, and the statement 'the model trained incrementally yields the same weights as that trained on both current and all previous data' does not follow for the actual algorithm. The paper neither acknowledges this mismatch nor provides a correction term; the step-invariant behavior in Table V could simply reflect small prompt drift rather than exact equivalence. Because the paper motivates PAL's anti-forgetting performance through this equivalence, the assumption is load-bearing, though the empirical gains may still hold.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PAL, an exemplar-free framework for multi-modal class-incremental learning in which one modality may be missing at training or test time. PAL combines two modules: modality-specific prompt pools that are tuned via backpropagation, and an analytic learning module in which the classifier is retrained by recursive least squares. The theoretical centerpiece is Theorem 1, which states that the RLS recursion yields the same classifier as joint training on all seen tasks. Experiments on UPMC-Food101 and N24News compare PAL with prompt-based and analytic baselines under missing rates from 10% to 90%, along with ablations of prompt structure, AL up-sampling, regularization, and large-step variants.","tokens_in":17954,"tokens_out":7379,"duration_ms":72328,"significance":"If the claims are established, PAL addresses a relevant and under-explored problem: multi-modal incremental learning under missing modalities, without rehearsal. The combination of prompt tuning with analytic learning is a sensible way to mitigate the under-fitting that arises when the backbone is frozen. The paper includes a self-contained derivation of the RLS recursion, broad experiments over missing-rate configurations, and ablations that isolate the prompt and AL components. The main reservation is that Theorem 1 is not actually satisfied by the proposed algorithm, because the prompt module—which is part of the feature extractor—is retrained at every increment, so the exact joint-training equivalence is not achieved. The empirical contribution is potentially strong, but the absence of error bars prevents significance claims.","major_comments":[{"comment":"Theorem 1 is derived for fixed feature matrices H_1,...,H_k (Appendix, Eqs. (1)-(15)), and the proof is algebraically correct under that assumption. However, in PAL the feature mapping is not fixed: Algorithm 1 step 4 updates P_v and P_t by backpropagation at every task, and Eq. (7) extracts H_k with the newly updated prompts, while the stored H_1,...,H_{k-1} come from earlier, different prompt states. Therefore Eqs. (13)-(14) do not minimize the joint objective Eq. (11) under the final feature mapping, and the claim after Theorem 1 that 'the model trained incrementally yields the same weights as that trained on both current and all previous data' is not justified by the proof. Since the paper uses this equivalence to explain anti-forgetting and calls Table V a demonstration of a 'step-invariant property,' this mismatch is load-bearing. Please restrict the theorem to the fixed-prompt condition and explicitly treat prompt drift as an approximation, or modify the algorithm so all historical features are produced by a single fixed mapping (e.g., freeze prompts after task 1, or recompute historical H_i after each prompt update if memory allows).","section":"III.D.2, Algorithm 1, Theorem 1"},{"comment":"The paper states that results are averaged over 3 runs but reports no standard deviations, confidence intervals, or significance tests. Several comparisons are close (e.g., Table I, eta=90% missing-both, PAL Acc 69.69 vs ACIL 65.93, while PAL FG 5.32 vs ACIL 4.33 is worse), so without variance information the claim that PAL 'significantly outperforms' all baselines is not supportable. Please report error bars or other variability measures and temper the significance language accordingly.","section":"IV.B.1, Tables I-IV"}],"minor_comments":[{"comment":"In Eq. (2), 'sim(·, ·) measures consine similarity' should be 'cosine similarity.'","section":"III.C"},{"comment":"The dataset name is written as 'UMPC-Food101' in some places and 'UPMC-Food101' elsewhere; please standardize.","section":"V.A"},{"comment":"Step 5 says 'Update R_k and W_k via AL using Eq. (13) and Eq. (14), respectively,' but Eq. (13) updates W_k and Eq. (14) updates R_k; the mapping in the sentence is reversed.","section":"Algorithm 1"},{"comment":"References [23] and [51] are the same DS-AL paper, which causes duplicate citation ambiguity. Also, reference [54] is attributed to Dosovitskiy et al. but lists the ViLT title; the citation should point to the Vision Transformer paper ('An Image is Worth 16x16 Words') for the statement in Section IV.B.1.","section":"References"},{"comment":"The manuscript is ambiguous about the state of the classifier when each new task begins. Section III.D.1 says the prompt module and classifier are trained via BP, but Algorithm 1 step 4 says only 'Update P_v and P_t via BP,' and step 5 re-trains the classifier via AL. Clarify whether the AL-trained classifier from step k-1 is used to initialize the BP stage at step k, or whether a fresh classifier is used, and how the two linear layers (W_FNN in Eq. (6) and the AL weights W_up and W) interact after re-training.","section":"III.D.1, Algorithm 1"},{"comment":"The conclusion that PAL achieves the lowest forgetting should be read against Table I: at eta=90% missing-both, ACIL reports FG 4.33 vs PAL 5.32. The text says 'lowest values across most settings,' which is careful, but the abstract and conclusion should not claim universal superiority on forgetting without qualification.","section":"V.A"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising empirical setup and a self-contained derivation of a standard RLS recursion, but the central theoretical claim is stated more strongly than the algorithm supports. The prompt module is retrained at each incremental step, so Theorem 1's fixed-feature assumption is violated by the reported method. This is fixable either by changing the algorithm to match the assumption or by explicitly characterizing the approximation error. The absence of error bars is a further obstacle to evaluating the empirical claims. The reliance on the authors' own prior analytic-learning work is noticeable but not inappropriate; the novelty lies in combining prompts with analytic learning for missing-modality multi-modal CIL, which is sufficient for a revision if the claims are made precise."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jian — quick take on 2501.09352. The useful part is real: PAL beats the strongest prior MMCIL baselines (RebQ, ACIL) by large margins on Food101 and N24News across many missing-modality settings, and the ablations show both the prompt module and the analytic-learning module earn their keep. The paper is clearly written, the experimental grid is wide, and the appendix proof is correct under its stated assumption.\n\nWhat's genuinely new is the integration: modality-specific prompt pools for missing-modality compensation, then an RLS classifier on frozen ViLT features. The RLS recursion is a re-derivation of ACIL/DS-AL, and the prompt reconstruction follows MAP/RebQ, so novelty is in the combination, not the components. That's fine. The 'first attempt' sentence in the conclusion is simply wrong, since the paper cites RebQ, which already tackles MMCIL with missing modality.\n\nThe main soft spot is the gap between Theorem 1 and Algorithm 1. The theorem assumes all H_i come from one fixed feature mapping, so that the RLS recursion exactly matches joint training. But step 4 of Algorithm 1 retrains the prompts via backpropagation at every task before extracting H_k. The stored H_1,...,H_{k-1} were produced by earlier prompt states, so the column blocks are not from the same function. Nothing in the paper acknowledges this, so the claimed exact equivalence to joint training is not actually satisfied by the method as run. The step-invariant behavior in Table V could just reflect small prompt drift. This is addressable: either state the theorem for a fixed extractor and treat prompt drift as an approximation, or add a correction. As written, the anti-forgetting justification is overweight on a theorem that doesn't strictly apply.\n\nMinor issues: results are 3-run averages with no error bars or standard deviations, and 'first attempt' should be softened.\n\nOverall: the empirical contribution is solid, the theory is overstated but the flaw is not a fatal one. The paper deserves a serious referee, and I'd ask the referee to focus on the fixed-feature assumption and on confirming the baselines use the same missing-modality protocol. Send it out.","headline":"PAL is a solid empirical MMCIL contribution with an overclaimed theoretical guarantee that only holds under a fixed feature extractor, which the method itself violates.","tokens_in":18478,"tokens_out":3168,"would_cite":false,"duration_ms":29336,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PAL combines modality-specific prompts with a recursive least-squares classifier so that multi-modal class-incremental learning stays accurate even when image or text is missing at train and test time.","keywords":["multi-modal class-incremental learning","missing modality","prompt learning","analytic learning","recursive least squares","catastrophic forgetting","exemplar-free continual learning","image-text learning"],"falsifier":"Run PAL on UPMC-Food101 with missing-both at 70% missing, then after each step compare the recursively updated classifier weights against the closed-form solution of Eq. (12) computed from all data seen so far; if the norm of the difference is not near machine precision, Theorem 1's premise fails under prompt retraining.","tokens_in":17459,"feed_emoji":"🧩","tokens_out":4088,"duration_ms":40573,"temperature":0.7,"pith_summary":"The paper claims that multi-modal class-incremental learning can be made robust to missing modalities by pairing modality-specific prompts with an analytic recursive least-squares classifier. It argues that prompts preserve a holistic representation when image or text is absent, and that the recursive update of the classifier is exactly equivalent to joint training on all data seen so far, so forgetting is controlled without storing exemplars. If true, continual learners on audio-visual or image-text streams could be deployed in realistic settings where sensors or inputs fail, rather than assuming every sample arrives complete.","feed_headline":"Missing a modality? PAL keeps multi-modal learning on track","feed_subtitle":"Prompt pools patch absent image or text while a recursive least-squares classifier cuts forgetting.","key_machinery":"The load-bearing object is the recursive least-squares classifier fed by a frozen multi-modal transformer backbone with modality-specific prompt pools. The RLS recursion, rooted in the Woodbury matrix identity, carries the anti-forgetting guarantee: if the feature mapping is fixed, incremental weight updates equal the weights from joint training on all past and current data, so old knowledge is preserved without exemplars. The prompt pools, one for image and one for text, selected by attention-weighted cosine similarity, do the complementary job of reconstructing missing-modality query embeddings and maintaining a holistic input representation, which addresses the under-fitting that a frozen analytic backbone would otherwise suffer on complex multi-modal data.","core_discovery":"The central discovery is that multi-modal class-incremental learning under missing modality can be reformulated as a Recursive Least-Squares problem and solved analytically. With the pre-trained multi-modal backbone frozen, the classifier at step k is updated as $\\hat{W}_k = \\hat{W}_{k-1} - R_k H_k^T H_k \\hat{W}_{k-1} + R_k H_k^T Y_k$, with $R_k$ updated by the Woodbury identity; Theorem 1 proves this recursion reproduces the joint-training solution. Modality-specific prompt pools, tuned by backpropagation with a reconstruction loss, compensate for absent inputs so the frozen backbone still sees a holistic representation. The paper reports that this combination beats prior prompt-based, replay-based, and analytic baselines on UPMC-Food101 and N24News across missing rates from 10% to 90%, with lower forgetting and accuracy that stays roughly constant as the number of incremental steps grows from 5 to 50.","pith_inferences":["The equivalence in Theorem 1 likely degrades when prompt pools shift features across tasks, since the recursion assumes a fixed feature mapping; a testable extension is to add a correction term to the RLS update when prompts evolve.","Because the classifier is analytic, per-task prompt retraining may need fewer epochs, and the reconstruction loss acts as a self-supervised regularizer that could transfer to uni-modal class-incremental learning.","The step-invariance result suggests the remaining bottleneck in long-run continual learning here is representation quality rather than storage, so a direct comparison against replay methods with large buffers would clarify where PAL's advantage comes from.","The same two-module design with additional prompt pools may extend to audio-visual pairs or tri-modal inputs, though the paper does not yet demonstrate that extension."],"forward_implications":["Deploying continual learners on real image-text or audio-visual streams where modalities drop out becomes feasible without storing old exemplars.","The method's accuracy and forgetting stay nearly constant from 5 to 50 incremental steps, so fine-grained class arrivals do not degrade old knowledge.","Missing rates up to 90% are handled: the paper reports 69.69% accuracy on UPMC-Food101 in the 90% missing-both scenario, with lower forgetting than competing baselines.","The framework removes the modality-completeness assumption that current multi-modal class-incremental methods rely on, opening the same design to other paired modalities.","The authors state tri-modal learning and beyond as the planned next step, suggesting the prompt-pool-plus-analytic-classifier structure is intended to generalize."],"supporting_citations":[{"why":"Supplies the frozen multi-modal transformer backbone whose joint embeddings feed the analytic classifier.","marker":"[31]"},{"why":"Provides the analytic class-incremental learning formulation and recursive least-squares update that PAL adapts.","marker":"[22]"},{"why":"Introduces the missing-modality prompting approach and evaluation protocol, including dummy inputs and missing-rate settings, that PAL extends to the incremental setting.","marker":"[36]"},{"why":"The direct multi-modal class-incremental baseline with missing modality that PAL compares against and outperforms.","marker":"[26]"},{"why":"Introduces prompt-pool prompting for continual learning on frozen backbones, the foundation for PAL's modality-specific pools.","marker":"[40]"},{"why":"Supplies the complementary task-invariant and task-specific prompt design used as a comparison baseline.","marker":"[41]"},{"why":"Provides the attention-weighted prompt selection mechanism used to compute the prompt weights in Eq. (2)-(3).","marker":"[50]"}],"fun_headline_variants":["PAL: Prompts plus recursive least squares fix missing modalities","Missing data? PAL's analytic update keeps incremental learning on track","PAL uses prompts and least squares to beat missing-modality forgetting","Exemplar-free PAL: Prompts and RLS handle missing modalities","PAL: Analytic solution for multi-modal incremental learning under missing data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The recursive classifier update is exact only if the frozen backbone produces the same features for a given input at every incremental step, yet PAL retrains the prompt pools with backpropagation at each task, so the feature map shifts and the claimed equivalence to joint training becomes approximate.","fun_headline_variants_meta":{"raw":{"variants":["PAL: Prompts plus recursive least squares fix missing modalities","Missing data? PAL's analytic update keeps incremental learning on track","PAL uses prompts and least squares to beat missing-modality forgetting","Exemplar-free PAL: Prompts and RLS handle missing modalities","PAL: Analytic solution for multi-modal incremental learning under missing data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000624,"raw_usage":{"total_tokens":2903,"prompt_tokens":973,"completion_tokens":1930,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":1840}},"tokens_in":589,"tokens_out":1930,"duration_ms":15804,"temperature":1.0,"reasoning_tokens":1840,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:06:22.677732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run PAL on UPMC-Food101 with missing-both at 70% missing, then after each step compare the recursively updated classifier weights against the closed-form solution of Eq. (12) computed from all data seen so far; if the norm of the difference is not near machine precision, Theorem 1's premise fails under prompt retraining.","supporting_citations":[{"cited_title":"Vilt: Vision-and-language transformer without convolution or region supervision,","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen multi-modal transformer backbone whose joint embeddings feed the analytic classifier."},{"cited_title":"Acil: Analytic class-incremental learning with absolute memorization and privacy protection,","cited_arxiv_id":null,"evidence_quote":"Provides the analytic class-incremental learning formulation and recursive least-squares update that PAL adapts."},{"cited_title":"Multimodal prompt- ing with missing modalities for visual recognition,","cited_arxiv_id":null,"evidence_quote":"Introduces the missing-modality prompting approach and evaluation protocol, including dummy inputs and missing-rate settings, that PAL extends to the incremental setting."},{"cited_title":"Learning to prompt for continual learning,","cited_arxiv_id":null,"evidence_quote":"Introduces prompt-pool prompting for continual learning on frozen backbones, the foundation for PAL's modality-specific pools."},{"cited_title":"Dualprompt: Complementary prompting for rehearsal-free continual learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the complementary task-invariant and task-specific prompt design used as a comparison baseline."},{"cited_title":"Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning,","cited_arxiv_id":null,"evidence_quote":"Provides the attention-weighted prompt selection mechanism used to compute the prompt weights in Eq. (2)-(3)."}],"review_version":1}