{"id":"c9f06dac-26da-41db-b50b-c22e565bcb26","arxiv_id":"2501.08878","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MSDEM grows per-task experts on top of multiple frozen ViT backbones with an attention-based fusion and a Gumbel-Softmax graph router, and reports state-of-the-art accuracy on multi-domain continual image classification.","lead":"A new continual learning method, MSDEM, uses several frozen pretrained vision transformers as backbones and grows a dedicated expert for each new task, fusing backbone features with attention and reusing past experts through a learned graph router. The authors report top average accuracy on multi-domain image-classification benchmarks while cutting trainable parameters by roughly 70 percent and training time by 89 percent versus a strong prompt-based baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MSDEM's state-of-the-art claim hinges on task identity being supplied at test time: Eq. (10) selects expert t by construction, yet no task-agnostic inference protocol is specified, so the comparison may be task-incremental rather than the stated class-incremental setting.","rationale":"I read the paper in good faith. The architecture is coherent and the ablations are informative; the parameter and time savings are real and independently documented in Table 3. My concern is not about internal consistency of the equations (they are consistent) but about whether the central claim—state-of-the-art performance in multi-domain continual learning—is supported under the setting the paper claims. The decisive condition is the inference protocol: because Eq. (10) always uses the current task's expert, the method is only executable when the task index is known. The problem statement explicitly says 'class-incremental learning paradigm' (Sec. 3.1), and class-incremental normally means no task label at test time. The manuscript never reconciles this. I agree with the reader that this is the weakest load-bearing assumption. The secondary issue (small absolute gaps vs. StarPrompt, relative error reduction of 16.98% across Table 1, within std dev) compounds it: even under the favorable protocol, the advantage is marginal; a protocol mismatch would erase it. Thus the correct verdict is CONDITIONAL: the paper should either define the evaluation as task-incremental and justify the comparison, or provide and evaluate a task-agnostic inference mechanism. My stress-test does not change the reader's conditional verdict; it sharpens the specific condition that must be verified.","tokens_in":23871,"tokens_out":4168,"duration_ms":45320,"concrete_test":"Obtain the evaluation script from the released repository (https://github.com/LexRider/MSDEM) and check whether test batches are ordered or deduplicated by task with a known current task index. Then run a task-agnostic variant: merge all task test sets, withhold task labels, and use either a logit-max criterion across all expert heads or a small task-ID classifier trained from the frozen backbone features. If the average over this pooled protocol falls below the reported ~98% or below StarPrompt's pooled accuracy, then the SOTA claim is conditional on task-incremental evaluation and the class-incremental claim in Sec. 3.1 is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 frames the problem as class-incremental learning over a cross-domain stream S. However, every prediction rule in the paper is task-indexed: Eq. (3) uses the j-th expert's classifier, and Eq. (10) uses the current task's expert E_t and router row M_t. The router is trained per task and is never used to select an expert for an unlabeled test sample. The evaluation section reports 'Average' accuracy across tasks but never states whether the test set provides task/domain identity or whether a single model must classify all classes without task labels. If the test protocol supplies the task index, MSDEM is evaluated in a task-incremental (multi-head) setting, which is an easier problem than the class-incremental setting claimed in the problem statement; if it does not, the architecture has no mechanism to choose E_t for a sample of unknown task. This distinction is decisive because the headline SOTA comparison is against StarPrompt, a prompt-based continual learner that is designed for class-incremental evaluation without task identity. The reported advantage of MSDEM2-3ep over StarPrompt is small (absolute differences of 0.0–0.5 percentage points in Table 1, mostly within one standard deviation), so a mismatch of evaluation protocol, rather than algorithmic merit, could plausibly explain the result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes MSDEM, a dynamic expansion method for multi-domain class-incremental learning. The model uses two or three frozen pre-trained ViT backbones to extract features, concatenates them, and creates a new expert for each task, where each expert contains a Dynamic Expandable Attention Mechanism (DEAM), an adaptive layer, a linear classifier, and a Dynamic Graph Weight Router (DGWR) based on a learned relation matrix and Gumbel-Softmax. Previous experts are frozen, and only the current expert's parameters are updated. Experiments cover six two-domain, two three-domain, and one four-domain scenarios, with comparisons against replay, prompt, and mixture-of-experts baselines, plus ablations and a computational-cost analysis. The paper claims state-of-the-art performance with fewer trainable parameters and less training time than StarPrompt.","tokens_in":24115,"tokens_out":7092,"duration_ms":69937,"significance":"If the claims hold, MSDEM would be a useful recipe for combining multiple frozen backbones in multi-domain continual learning, and the released code and the breadth of the experimental matrix are strengths. The computational-cost comparison is also informative. However, the significance is currently conditional: the evaluation protocol is not specified and appears to assume task identity at test time, so the comparison against class-incremental baselines such as StarPrompt is not yet established on equal footing. The headline '16.98% improvement' is a relative error reduction, not an absolute accuracy gain, which further weakens the strength of the claimed margin.","major_comments":[{"comment":"The paper does not specify an inference protocol for unlabeled test samples, and every prediction rule is task-indexed. The problem statement says the model is evaluated on all testing datasets when training is finished, but it never states whether each test set comes with its task/domain index. Equations (3) and (10) use the j-th or t-th expert by construction, and the router in Eq. (6) is trained per task and is never used to choose an expert for a sample of unknown origin. If test-time task labels are assumed, the evaluation is task-incremental (multi-head) rather than the class-incremental setting claimed in Section 3.1, and the comparison with StarPrompt, which operates without task labels, is not on equal footing. If task labels are not assumed, the architecture has no mechanism to select among the per-task classifiers. Please state the exact evaluation protocol, provide task-agnostic results, or narrow the claim to task-incremental learning.","section":"Section 3.1, Eqs. (3) and (10)"},{"comment":"The headline '16.98% improvement in the Average metric' is a relative error reduction, not an absolute accuracy improvement. Averaging the six dual-domain scenarios in Table 1, MSDEM2-3ep reaches about 98.68% average accuracy versus about 98.42% for StarPrompt, an absolute gap of about 0.27 points; the improvement as a fraction of StarPrompt's error (about 1.58 points) is roughly 17%, which matches the quoted figure. The per-scenario absolute differences in Table 1 are 0.0–0.5 points and mostly lie within one standard deviation. The text should explicitly say 'relative error reduction' and should report per-scenario absolute differences and a significance check alongside the aggregate percentage.","section":"Section 4.2, Table 1"},{"comment":"The caption of Table 1 in the appendix first says that the MSDEM2 models use ViT 1 and ViT 2 as backbones and then says that the results reported in the paper for MSDEM2 correspond to the configuration using ViT 1 and ViT 3; this is contradictory. Combined with the implementation list in Section 4.1, which does not unambiguously map 'ViT 1' to 'fine-tuned on ImageNet-1K', the reader cannot tell which backbone pair produced Tables 1 and 2 in the main text or which configuration the ablations refer to. Please reconcile the naming and state, for every table, exactly which two or three backbones are used.","section":"Appendix C.1, Table 1 caption"}],"minor_comments":[{"comment":"The set-builder expression and the subsequent summation contain mismatched indices: it should be ˜{z}_j = ¯{z}_j cM_t[j] and Z_t = Σ_{j=1}^t ¯{z}_j cM_t[j], rather than the current ˜z_c / ˜z_t notation.","section":"Algorithm 1, Step 3"},{"comment":"The expression 'z1 ⊗ z2⊗, · · ·, ⊗ zt′' has a stray comma and should define ⊗ explicitly as concatenation along the feature dimension.","section":"Eq. (2)"},{"comment":"The dataset configuration section gives the task-to-domain mapping only for the T-C100-B-C10 scenario; the class splits and their order for the other nine scenarios are not specified, which impedes exact reproduction.","section":"Appendix B.3"},{"comment":"There are numerous spelling and grammar issues, including 'behavrious' in Section 3.3, 'permutating' in Section 4.3, and 'A verage' in Table 2; a careful copyedit is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The central uncertainty is whether the authors can provide a task-agnostic evaluation. If they cannot, the paper should be reframed as task-incremental, and the SOTA claim against StarPrompt should be dropped or substantially qualified. The relative-error phrasing is likely to mislead readers and should be corrected regardless. I would support publication after these issues are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is a credible empirical contribution to multi-domain continual learning. The idea is to take several frozen ViT backbones, concatenate their features, and for each new task build a small expert with an attention module and a Gumbel-Softmax router that reweights previous experts. The experiments are broad: six dual-domain, two three-domain, one four-domain, nine scenarios, with baselines, ablations, forgetting curves, and a useful cost comparison. The authors ship code. That part is genuinely useful.\n\nThe soft spot is the evaluation protocol. The paper frames the problem as class-incremental, but every prediction rule—Eq. (3), Eq. (10)—uses the task-specific expert and router row. There is no mechanism to select an expert for an unlabeled test sample, and the paper never states whether the test set provides task/domain identity. If it does, this is task-incremental learning, which is easier than class-incremental and is not what the abstract claims. The comparison with StarPrompt is especially problematic because StarPrompt is a prompt-based method that handles class-incremental evaluation without task labels. The stress-test concern lands.\n\nThe headline '16.98% improvement' is a relative error reduction; absolute differences over StarPrompt are under one point and often within standard deviation. That claim is overstated. The triple-backbone variant is worse than the dual on several tasks, and the ablations show that the attention and router sometimes hurt, which is honest but complicates the narrative. The router appears to be used only during training, not at test time, so the 'dynamic graph weight router' is a training-time mechanism, not an inference-time selector.\n\nThat said, the work is not flawed at the core. The architecture is reasonable, the experiments are reproducible in principle, and the cost analysis shows a real efficiency advantage. What's missing is clarity about the setting and a more measured summary of the results.\n\nI'd send it to a referee if the authors can clarify the inference protocol and either re-run the comparison in a genuinely class-incremental setting or honestly label it task-incremental. As it stands, I wouldn't cite it, but I'd be curious to see the revision. Reading group: maybe, if the discussion centers on evaluation protocols.\n\nRecommendation: engage, ask for a revision that resolves the protocol ambiguity and tempers the claims.","headline":"A useful empirical study of multi-backbone dynamic expansion for multi-domain continual learning, but the SOTA claim rests on an unstated task-inference protocol and relative error numbers that flatter the absolute gains.","tokens_in":24710,"tokens_out":2811,"would_cite":false,"duration_ms":28226,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A continual-learning model built from frozen vision transformers plus a learned expert router reports higher average accuracy than replay, prompt, and mixture-of-experts baselines on multi-domain class-incremental streams, with far fewer…","keywords":["continual learning","multi-domain learning","dynamic expansion","mixture of experts","Gumbel-Softmax router","vision transformer","class-incremental learning","catastrophic forgetting"],"falsifier":"Rerun the six dual-domain streams with the same training protocol but at test time give each sample no task or domain label and force the model to select an expert from the input alone (for example, choose the expert whose classifier has the highest max softmax score). If the Average metric drops materially below the table values, or if the model cannot be run at all without the task oracle, then the reported state-of-the-art result does not extend to task-agnostic multi-domain continual learning.","tokens_in":23606,"feed_emoji":"🔄","tokens_out":6077,"duration_ms":59672,"temperature":0.7,"pith_summary":"MSDEM is a continual-learning model for streams that shift in both class and data domain. Its central claim is that a new per-task expert built on several frozen pre-trained ViT backbones, recombining their features with attention and a Gumbel-Softmax graph router, learns each new domain well without replay or prompt tuning. In the paper's experiments the two-backbone version reports average accuracy at or above StarPrompt's in all nine domain-stream configurations while cutting trainable parameters by about 70% and training time by about 89% in the dual-backbone setup. A sympathetic reader would take away that multi-domain continual learning can be treated as a feature-recombination problem over a fixed pool of extractors.","feed_headline":"MSDEM tops nine multi-domain continual-learning streams","feed_subtitle":"Frozen ViT backbones plus a learned router beat replay and prompt baselines.","key_machinery":"The machinery that carries the claim is the combination of a frozen multi-backbone feature pool, a per-task Dynamic Expandable Attention Mechanism (DEAM), and a Dynamic Graph Weight Router (DGWR). DGWR models the experts as nodes of a graph: row $t$ of a relation matrix $C$ is turned into a weight vector by a Gumbel-Softmax draw with Gaussian noise and a temperature $\\tau$, so the router can interpolate between hard one-hot expert selection and soft weighting of all experts. The router's learned weights combine normalized representations of previous experts with the current expert's feature, and a graph-attention block then forms the final representation for the current classifier. Only the current task's attention matrices, router row, embedding, and classifier are updated; all prior parameters are frozen.","core_discovery":"The core discovery, on the paper's own terms, is that freezing multiple ViT backbones and dynamically growing a lightweight expert per task yields better multi-domain continual learning than replay-based, prompt-based, or mixture-of-experts baselines. Each expert consists of a Dynamic Expandable Attention Mechanism (DEAM), which reweights the concatenated output of the backbones for the new task, an adaptive embedding, and a linear classifier. The Dynamic Graph Weight Router (DGWR) keeps a relation matrix over experts, converts its current row into selection weights via Gumbel-Softmax, and uses those weights to combine previous experts' representations with the new expert's; only the current row is trained. The paper reports that MSDEM2 trained for three epochs reaches average accuracy 98.1-99.7 on the dual-domain streams (Table 1), 97.59-98.15 on the three- and four-domain streams (Table 2), and reduces trainable parameters by 70.36% and training time by 89.35% relative to StarPrompt-2nd.","pith_inferences":["The paper evaluates under an implicit task oracle: each test sample's domain is known, so the matching expert and router row are used. A task-agnostic version, where the model must infer the expert from the input, is not trained or evaluated; that is the most direct extension to test.","Because the router and attention operate on feature vectors rather than pixels, the same scheme could be transplanted to other input types (audio, text, sensor streams) by swapping the frozen backbones for frozen encoders.","The reported asymmetric domain weights imply a concrete prediction: permuting the order of the same two domains should flip which domain receives the larger router weight, and the paper's heatmaps already show such order dependence; a quantitative test would compare router rows across all 16 permutations.","A held-out generalization test—introducing a domain whose images are unlike any pretraining source—would separate true domain transfer from recombining familiar feature spaces."],"forward_implications":["Multi-domain class-incremental streams can be learned without a memory buffer or generative replay, because freezing old expert parameters and adding only a small per-task expert prevents forgetting.","The dual-backbone configuration beats the triple-backbone one on most average scores, so more frozen feature extractors do not automatically improve continual learning and the choice of backbones matters more than their count.","With three training epochs, MSDEM2 reaches average accuracy at or above StarPrompt in every reported stream, indicating the gains come from dynamic multi-source expansion rather than from using more training passes.","The router's learned weights are asymmetric and order-dependent across domain pairs, so the amount of knowledge transferred between two domains changes with which domain is learned first."],"supporting_citations":[{"why":"Supplies the Vision Transformer backbone used as the frozen feature extractors that all experts are built on.","marker":"[12]"},{"why":"StarPrompt is the main prompt-based baseline whose average accuracy forms the comparison ceiling in Tables 1 and 2.","marker":"[33]"},{"why":"Mixture-of-experts adapter baseline used to bound the comparison for dynamic expansion methods.","marker":"[50]"},{"why":"Random-projection and pre-trained-model baseline used for accuracy and forgetting-rate comparisons.","marker":"[32]"},{"why":"Replay-based baseline whose variants DER++ and DER+++refresh are compared in the multi-domain scenarios.","marker":"[5]"},{"why":"Data-augmentation prompt baseline compared across the dual-, three-, and four-domain task configurations.","marker":"[22]"}],"fun_headline_variants":["Dynamic expansion beats replay and prompts in multi-domain learning","MSDEM: Freeze backbones, grow experts for new domains","Multi-domain continual learning: fewer parameters, faster training","Graph router reuses experts to top multi-domain streams","Growing experts on frozen backbones masters multi-domain streams"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model assumes each test sample comes with its task or domain identity so the right expert and classifier can be chosen; without that identity, the paper provides no mechanism to decide which expert should make the prediction.","fun_headline_variants_meta":{"raw":{"variants":["Dynamic expansion beats replay and prompts in multi-domain learning","MSDEM: Freeze backbones, grow experts for new domains","Multi-domain continual learning: fewer parameters, faster training","Graph router reuses experts to top multi-domain streams","Growing experts on frozen backbones masters multi-domain streams"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000784,"raw_usage":{"total_tokens":3452,"prompt_tokens":930,"completion_tokens":2522,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":2443}},"tokens_in":546,"tokens_out":2522,"duration_ms":16846,"temperature":1.0,"reasoning_tokens":2443,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:16:19.445705+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the six dual-domain streams with the same training protocol but at test time give each sample no task or domain label and force the model to select an expert from the input alone (for example, choose the expert whose classifier has the highest max softmax score). If the Average metric drops materially below the table values, or if the model cannot be run at all without the task oracle, then the reported state-of-the-art result does not extend to task-agnostic multi-domain continual learning.","supporting_citations":[{"cited_title":"Semantic Residual Prompts for Continual Learning","cited_arxiv_id":"2403.06870","evidence_quote":"StarPrompt is the main prompt-based baseline whose average accuracy forms the comparison ceiling in Tables 1 and 2."},{"cited_title":"Boosting continual learning of vision-language models via mixture-of-experts adapters","cited_arxiv_id":null,"evidence_quote":"Mixture-of-experts adapter baseline used to bound the comparison for dynamic expansion methods."},{"cited_title":"McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel","cited_arxiv_id":null,"evidence_quote":"Random-projection and pre-trained-model baseline used for accuracy and forgetting-rate comparisons."},{"cited_title":"Dark experience for general continual learning: a strong, simple baseline","cited_arxiv_id":null,"evidence_quote":"Replay-based baseline whose variants DER++ and DER+++refresh are compared in the multi-domain scenarios."},{"cited_title":"Generating instance-level prompts for rehearsal-free continual learning","cited_arxiv_id":null,"evidence_quote":"Data-augmentation prompt baseline compared across the dual-, three-, and four-domain task configurations."}],"review_version":1}