{"id":"9b7ffe31-2dc7-4591-9193-1ef4aa9fec26","arxiv_id":"2505.23271","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LADA adds lightweight label-specific memory vectors to a frozen CLIP image encoder, removing the need for task-parameter selection and reporting state-of-the-art X-TAIL benchmark results.","lead":"LADA is a new adapter for CLIP that helps it keep learning new image classification tasks without forgetting old ones: it stores a small set of per-class memory vectors instead of selecting among task-specific modules. The paper reports higher accuracy than previous continual learning methods on a 10-dataset benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim is not falsifiable as reported: the fusion weight for seen-class logits is never quantified, and Transfer comparisons against RAIL are excluded under a protocol difference, so the headline margins in §4.1 depend on unstated choices.","rationale":"The reader's weakest assumption identifies the same two load-bearing issues: the unquantified fusion weight and the excluded RAIL Transfer results. My stress-test agrees that both are central and that a CONDITIONAL verdict is appropriate. I could not find a more damaging internal inconsistency: the method description is coherent, the ablations in Table 3 attribute gains to each component in the right direction, and the per-task tables in §D are internally consistent with the reported averages. The main weakness is that the headline SOTA claim cannot be verified without the missing hyperparameter and without a symmetric baseline protocol. I deliberately did not raise the absence of error bars as the primary concern because, while real, it is secondary to the missing fusion weight: even with error bars, the unreported weight prevents reproduction. I also did not treat the 'Transfer surpasses zero-shot' result as a red flag, since the paper gives a plausible mechanism (task recall) and Figure 3 supports it internally. Agreement with the reader is 'agree' because our conclusions coincide on the specific conditional items needed for acceptance.","tokens_in":21096,"tokens_out":1956,"duration_ms":18477,"concrete_test":"Obtain the released code (github.com/MaolinLuo/LADA) and (i) locate the fusion weight for the seen-class linear combination of LADA and text logits; if it is a single fixed constant, re-run Table 1 with the weight set to 0 and to 1 and report Transfer/Average/Last for both extremes; if the resulting Average/Last range overlaps the RAIL row (71.3/82.3), the SOTA claim in §4.1 is not robust. (ii) Re-run RAIL under the no-selector protocol by appending its trained classifier logits to all seen and unseen classes without zero-shot gating, and recompute Transfer; if RAIL's Transfer then exceeds LADA's 61.5 (16-shot) or 61.9 (full-shot), the state-of-the-art Transfer claim fails under symmetric evaluation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that LADA achieves state-of-the-art Transfer/Average/Last on X-TAIL (§4.1). Two unquantified choices could change whether that claim holds. (1) In §3.2 'Overall Framework', the final prediction for seen classes uses 'a linear weighting' between LADA logits and text-feature logits. The weight is never specified, not in §B (additional implementation details) nor in any ablation. If the weight was selected on X-TAIL test/validation data, the comparison to fixed-protocol baselines is informal. Since the text-only framework (BF) already achieves 59.4/70.9/82.1 (Table 3) and LADA alone adds 1.8/1.4/0.9, the convex combination's value determines how much of the gain is attributable to LADA versus the text encoder; without it, the reported 2.5/1.4/0.8 improvement over the best baseline cannot be reproduced or audited. (2) RAIL, the strongest baselines (Average/Last), are removed from the Transfer comparison entirely (§4.1), with the justification that RAIL's Transfer uses zero-shot CLIP as a selector. That is a reasonable methodological point, but it means the 'state-of-the-art' Transfer claim is established only against weaker baselines; a fair comparison would require running RAIL under the same no-selector protocol, or reporting both variants. The paper's Figure 3 shows LADA's task-recall without a selector is higher than with one, but it does not provide RAIL's equivalent task-recall under the same protocol. Thus the exclusivity of both headline margins is fragile: the reported numbers are conditional on an undisclosed fusion weight and an asymmetric baseline protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LADA, a label-specific CLIP adapter for cross-domain task-agnostic incremental learning (X-TAIL). LADA attaches per-class k-means cluster centers as lightweight memory units after a frozen CLIP image encoder, updates only the current task's units, and uses GMM-based augmented prototypes of previous classes plus feature distillation to mitigate backward and forward forgetting. A fine-tuned text encoder is jointly optimized, and inference combines LADA logits with text-feature logits for seen classes. On the X-TAIL benchmark under 16-shot and full-shot settings, the paper reports improvements in Transfer, Average, and Last over prior methods, with gains of 2.5/1.4/0.8 percentage points in the 16-shot setting and 2.9/2.4/2.9 in the full-shot setting.","tokens_in":21513,"tokens_out":4682,"duration_ms":51845,"significance":"If the reported results hold, LADA is a clean and practical contribution: it avoids inference-time prompt or adapter selection, requires no gradient flow to the CLIP image encoder, and scales parameter growth linearly with classes. The paper's strengths include a publicly available implementation, ablations that support the role of LADA and distribution-preserving training, and evaluation on a held-out benchmark with standard metrics. The core ideas—label-specific memory units and prototype distillation—are not entirely new in isolation, but their combination in a selector-free inference scheme is sensible, and the efficiency analysis is useful. The empirical case, however, rests on several unquantified choices and an asymmetric baseline comparison, so the state-of-the-art claim needs additional validation before it can be accepted at face value.","major_comments":[{"comment":"The inference-time fusion between LADA logits and text-feature logits for seen classes is never specified. The text says only that the final prediction is obtained by 'applying a linear weighting between the logits produced by LADA and the corresponding text features,' with no equation, numerical value, or sensitivity analysis. Since the text-only BF baseline already reaches 59.4/70.9/82.1 in the 16-shot setting (Table 3), the magnitude of the reported gains depends directly on this weight; if the weight was tuned on the X-TAIL test or validation splits, the comparison to fixed-protocol baselines is informal. Please report the exact fusion rule and an ablation over the weight. In the same passage, the two-stage procedure 'if the predicted class belongs to unseen classes' is ambiguous: it is not clear how the initial prediction that decides between text-only and weighted logits is made.","section":"§3.2 (Overall Framework)"},{"comment":"RAIL's Transfer results are omitted because RAIL uses a zero-shot CLIP selector, and the paper states that reported RAIL Transfer numbers directly use vanilla CLIP zero-shot accuracy. This is a legitimate protocol concern, but it means the headline Transfer improvement (2.5% in 16-shot and 2.9% in full-shot over the best listed baseline) is established only against methods that do not use the selector. To support a state-of-the-art claim for Transfer, the authors should either run RAIL under the same no-selector protocol or explicitly scope the claim to selector-free methods. Figure 3 provides LADA's task-recall with and without the selector but does not provide RAIL's task-recall under the same protocol, so it does not resolve the comparison.","section":"§4.1 (Main Results)"},{"comment":"All reported numbers are single runs with no error bars or seed information. The headline margins are small—0.8 percentage points in 16-shot Last and 1.4 in 16-shot Average—so run-to-run variance could change which method is best. Please report mean and standard deviation over at least three random seeds, and state whether hyperparameters (including β in Eq. (6) and the fusion weight) were selected on validation data or on the benchmark test splits.","section":"§4.1 and 'Implementation Details'"},{"comment":"The sharpness parameter β in Eq. (6) is described as modulating the transformation ϕ but its value is never given in the implementation details or in Appendix B. Because ϕ defines the mapping from inner products to logits and directly affects the loss in Eqs. (7), (8), and (10), the method is not fully reproducible without this value. Please report β and, if it varies across tasks or settings, describe how it is set.","section":"Eq. (6) and 'Implementation Details'"}],"minor_comments":[{"comment":"The checkmark-only table does not explicitly label which rows correspond to BF+LADA versus BF+DPT; the prose order implies a specific pairing, but the table should name each configuration to avoid ambiguity.","section":"Table 3"},{"comment":"In Appendix B, the AdaptFormer formula uses X and X′ without defining their dimensions or distinguishing the input to the first layer from the output of the multi-head attention; please clarify the notation.","section":"Eq. (15)"},{"comment":"The two panels use inconsistent legends: panel (a) shows 'w/o Selector' and 'w Selector' curves, while panel (b) shows only 'Acc Diff.'; please make the legends and axis labels uniform across panels.","section":"Figure 3"},{"comment":"The asterisk on λ1=32 is explained only in the body text, not in the table caption or footnote; please move the explanation into the table caption.","section":"Table 4"},{"comment":"The dataset name is written as 'Sun397' in at least one place in §4.5 and as 'SUN397' elsewhere; please standardize the capitalization.","section":"Abstract and §4.5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a conventional empirical continual-learning submission with a clear central claim. The main risk is not novelty but verifiability: the cross-method comparison depends on an unreported fusion weight and an omitted RAIL Transfer baseline, and the reported margins are small relative to the absence of error bars. If the authors supply the missing values and either run RAIL under a matched protocol or clearly limit the Transfer claim, the paper could become acceptable. I see no citation or scope issue that would require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about this paper. LADA is a genuine new mechanism—per-class cached memory vectors on the frozen CLIP image encoder, initialized by k-means and refined with GMM-augmented prototype distillation—and it removes the error-prone inference-time parameter selection that prompt/MoE methods require. The ablations show each component earns its place, and the paper ships code. The main caveat is not the method but the reporting: the linear fusion weight between LADA and text logits for seen classes is never quantified in §3.2 or §B, and β (logit sharpness, Eq. 6) is also unreported. Without those, the headline margins in §4.1 cannot be reproduced from the text. There are also no error bars; all results are single runs, which matters on a 10-task benchmark where baselines can shift by a point or two.\n\nOn the stress-test note: the first concern is legitimate and matches what I just said. The second concern, about excluding RAIL from Transfer, is weaker than it looks. RAIL's Transfer is definitionally zero-shot accuracy because RAIL uses zero-shot CLIP as a selector; comparing against it would not measure forward forgetting of the trained model. The paper's justification for exclusion holds. A fair comparison would require running RAIL without the selector, which is a reasonable request but more of a thoroughness point than a flaw.\n\nWhat the paper does well: it identifies a real failure mode in parameter-selection-based CLIP continual learning and offers a simple, scalable alternative that also handles forward forgetting well—Transfer sometimes exceeds vanilla zero-shot CLIP. The ablation in Table 3 and the hyperparameter analysis in Table 4 are informative, and results are reported under two task orders in both 16-shot and full-shot settings.\n\nWho this is for: people working on CLIP continual learning, prompt-based methods, or parameter-efficient adaptation. It is a solid empirical contribution. If I were refereeing, I would ask for the fusion weight and β, a sensitivity check on those, and error bars on the main tables—all easy fixes. With those, the SOTA claim would be properly auditable. The paper deserves a serious referee and, likely, acceptance after minor revision.","headline":"Solid new mechanism with a reproducibility gap: the missing fusion weight and β make the exact SOTA margins unverifiable, but the core claim holds and the paper deserves careful review.","tokens_in":22022,"tokens_out":4505,"would_cite":true,"duration_ms":46801,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Per-class vectors keep CLIP learning without task selectors","keywords":["continual learning","CLIP","label-specific adapter","vision-language models","catastrophic forgetting","cross-domain task-agnostic incremental learning","parameter-efficient fine-tuning","prototype distillation"],"falsifier":"Sweep the weighting hyperparameter over a grid on X-TAIL and check whether the reported Transfer, Average, and Last numbers require a value selected with test-set labels; also re-run RAIL with Transfer reported under the same protocol and see whether LADA's 2.5-2.9% Transfer advantage persists.","tokens_in":20947,"feed_emoji":"🧠","tokens_out":4750,"duration_ms":43606,"temperature":0.7,"pith_summary":"This paper introduces LADA, an adapter that lets a frozen CLIP model learn new tasks continually without choosing among task-specific parameters at inference time. Instead of partitioning parameters across tasks, LADA attaches a small set of learnable memory vectors to each class and classifies by taking inner products between the frozen image representation and these label-specific vectors. Old class vectors are frozen and new ones are added as tasks arrive, while feature distillation over cluster-center prototypes of previous classes protects old knowledge. The authors claim this design reduces both forward and backward forgetting and achieves state-of-the-art Transfer, Average, and Last accuracies on the 10-dataset X-TAIL benchmark in both 16-shot and full-shot settings.","feed_headline":"Per-class vectors keep CLIP learning without task selectors","feed_subtitle":"LADA freezes old class memories, distills old features, and beats prior methods on the 10-domain X-TAIL benchmark.","key_machinery":"The central object is the label-specific memory unit: for each class j of task k, a set of λ1 vectors W^k_j initialized by k-means cluster centers of that class's CLIP image features. The image representation is mapped to label-specific features by φ(i) = [W_1 i, ..., W_M i], and a fixed head outputs exp(−β(1−x)) for each class, acting like a nearest-neighbor classifier. Old units are frozen and new units are added per task, while distribution-preserved training fits a Gaussian mixture model to each old class and samples augmented prototypes to compute the distillation loss. The overall framework sums the LADA logits with the text-encoder logits, applying a linear weighting for seen classes during inference.","core_discovery":"LADA claims that continual learning with vision-language models need not rely on inference-time selection of prompts or adapters, and that per-class memory vectors appended to a frozen CLIP image encoder are enough to achieve state-of-the-art results. Each class is represented by k-means-initialized vectors, and the classifier is a fixed nearest-neighbor-style map using an exponential kernel on inner products. Training freezes the vectors of previously seen classes, updates only the new task's vectors, and uses GMM-augmented prototypes of old classes in the loss to preserve their feature distribution. On X-TAIL, the reported gains over the previous best approach are 2.5% Transfer, 1.4% Average, and 0.8% Last in the 16-shot setting, and 2.9%, 2.4%, and 2.9% in the full-shot setting.","pith_inferences":["The linear weighting between LADA logits and text-feature logits for seen classes is described but never quantified; reporting that value and its sensitivity would let others reproduce the exact numbers and test whether it transfers across benchmarks.","Because LADA is independent of the image encoder's internal structure, the same per-class memory mechanism could be applied to other frozen foundation encoders beyond CLIP.","The GMM-based prototype augmentation could be lifted out as a general replay-free forgetting defense for other continual classifiers.","It would be worth testing LADA on longer task sequences and open-set settings to see whether the growth of memory units and the seen/unseen decision rule hold up."],"forward_implications":["LADA removes the error-prone parameter-selection step of prompt- and MoE-based methods, so inference is a single forward pass over a unified feature space.","Because only the current task's label-specific vectors are updated and CLIP is frozen, training cost and memory grow only gradually as tasks accumulate.","Feature distillation with GMM-augmented prototypes preserves old-class distributions, mitigating both backward forgetting and forward forgetting of pretrained knowledge.","On several X-TAIL tasks, LADA's Transfer accuracy exceeds that of vanilla zero-shot CLIP, indicating that learned knowledge can improve classification of unseen classes.","The method is effective in both the 16-shot and full-shot settings, with larger margins in full-shot."],"supporting_citations":[{"why":"Supplies the X-TAIL benchmark, task order, and the RAIL baselines that LADA compares against.","marker":"(Xu et al., 2024)"},{"why":"Defines the Transfer, Average, and Last metrics and the zero-shot distillation approach LADA builds on.","marker":"(Zheng et al., 2023)"},{"why":"Provides the frozen CLIP image and text encoders whose features LADA adapts.","marker":"(Radford et al., 2021)"},{"why":"MoE-Adapters is the strongest prior adapter-based method that LADA outperforms without parameter selection.","marker":"(Yu et al., 2024)"},{"why":"AdaptFormer is the parameter-efficient fine-tuning module used to adapt the text encoder in LADA's framework.","marker":"(Chen et al., 2022)"}],"fun_headline_variants":["Per-class memories let CLIP keep learning indefinitely","LADA: label-specific vectors for forget-free CLIP training","No task selectors: frozen CLIP plus per-class vectors wins","One vector per class keeps CLIP lifelong learning","LADA: scalable continual learning with frozen CLIP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume the linear weight between LADA logits and text-feature logits for seen classes is a fixed hyperparameter rather than a value tuned on the X-TAIL test sets, and that the baselines are directly comparable under a shared inference protocol.","fun_headline_variants_meta":{"raw":{"variants":["Per-class memories let CLIP keep learning indefinitely","LADA: label-specific vectors for forget-free CLIP training","No task selectors: frozen CLIP plus per-class vectors wins","One vector per class keeps CLIP lifelong learning","LADA: scalable continual learning with frozen CLIP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000361,"raw_usage":{"total_tokens":1937,"prompt_tokens":918,"completion_tokens":1019,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":940}},"tokens_in":534,"tokens_out":1019,"duration_ms":8206,"temperature":1.0,"reasoning_tokens":940,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:49:18.198800+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Sweep the weighting hyperparameter over a grid on X-TAIL and check whether the reported Transfer, Average, and Last numbers require a value selected with test-set labels; also re-run RAIL with Transfer reported under the same protocol and see whether LADA's 2.5-2.9% Transfer advantage persists.","supporting_citations":[{"cited_title":"Preventing zero-shot transfer degradation in continual learning of vision-language models","cited_arxiv_id":null,"evidence_quote":"Defines the Transfer, Average, and Last metrics and the zero-shot distillation approach LADA builds on."},{"cited_title":"W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al","cited_arxiv_id":null,"evidence_quote":"Provides the frozen CLIP image and text encoders whose features LADA adapts."},{"cited_title":"Adaptformer: Adapting vision transformers for scalable visual recognition","cited_arxiv_id":null,"evidence_quote":"AdaptFormer is the parameter-efficient fine-tuning module used to adapt the text encoder in LADA's framework."}],"review_version":1}