{"id":"f9f5283b-fda5-41d6-b6d5-0fcac3155415","arxiv_id":"2412.18177","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"S6MOD, a state space model branch with a class-conditioned mixture of discretization and a contrastive discretization loss, modestly improves average accuracy and forgetting of seven online continual learning baselines.","lead":"This paper adds a plug-in module, S6MOD, to existing online continual learning systems and reports small accuracy gains across CIFAR-10, CIFAR-100, and Tiny-ImageNet. The module lets a state space model choose among several discretization patterns based on class uncertainty, aiming to learn new tasks without forgetting old ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ablation attribution: the headline gains are mostly from the auxiliary SSM branch plus distillation, not from the novel routing and contrastive loss; this undercuts the central mechanistic claim.","rationale":"The paper's central claim is that S6MOD's mixture of discretization and class-conditional routing improve OCL adaptability. Table 3 is the only ablation isolating these components, and it shows that a plain SS2D branch with an ETF classifier and KL distillation accounts for most of the gain. The routing-only and LCont-only ablations add only 0.31 and 0.32 points, within the reported standard deviations, and the combined 0.96-point gain is borderline. Without paired significance tests or a matched fixed-discretization control, the observed improvements are equally consistent with the hypothesis that any auxiliary branch with distillation provides the benefit. This threatens the internal validity of the paper's main mechanistic contribution. The reader's prototype-noise concern is real but secondary: even an imperfect router could yield the small marginal gains, whereas the attribution issue determines whether the claimed mechanism is credible. The right response is to require the control experiment and significance reporting, which matches the existing CONDITIONAL verdict, so no change to the reader's verdict is needed.","tokens_in":17499,"tokens_out":13406,"duration_ms":121100,"concrete_test":"Run the Table 3 'branch' control (SS2D core + ETF classifier + L_Diff, no routing, no LCont) against full S6MOD on all three datasets, with matched parameter counts and compute, and report paired significance across at least 10 seeds. If the full method does not beat the branch control by a statistically significant margin on most settings, the paper's mechanistic claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that S6MOD's class-conditional mixture of discretization and contrastive loss drive the reported adaptability gains. The paper's own ablation (Table 3, CIFAR-100 M=2k) places this premise in doubt: replacing the S6MOD core with plain SS2D plus the ETF-classifier KL distillation ('branch') already accounts for 3.28 of the 4.24-point gain over OnPro (33.33 to 36.61 vs 33.33 to 37.57). Adding class-conditional routing alone yields +0.31, adding LCont alone +0.32, and both together +0.96, with reported standard deviations of 0.72-1.04. No paired significance tests are provided, so the marginal effects of the two headline innovations (Eqs. 9-10 and Eq. 11) are not established as reliable. The experimental record is therefore consistent with a simpler alternative: a generic auxiliary SSM branch with distillation improves OCL, while the specific routing and contrastive mechanisms contribute little beyond it. Since the abstract credits the mixture-of-discretization/routing design for the gains, this attribution is load-bearing and currently unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes S6MOD, a plug-and-play branch for online continual learning (OCL) methods. The branch consists of a selective state space model (S6) with a mixture of discretization, a class-conditional routing mechanism that selects the number of discretization patterns per class based on prototype-margin uncertainty, and a contrastive discretization loss. The branch is supervised by a fixed ETF classifier and combined with the base method through a KL-divergence distillation loss. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet with seven baselines (ER, OCM, OnPro, OCM-CCLDC, OnPro-CCLDC, MOSE, MOE-MOSE) report improved average accuracy and lower average forgetting in many settings, with particular gains for OnPro and OCM. The paper claims that the module 'can be integrated into most existing methods and directly improve adaptability' and that the routing and contrastive loss are responsible for the gains.","tokens_in":17677,"tokens_out":5520,"duration_ms":47799,"significance":"The manuscript addresses a relevant problem—improving the adaptability of online continual learners—and proposes a modular solution that is easy to graft onto existing methods. The mathematical formulation of the mixture of discretization is standard selective SSM machinery, and the pseudo-code in Algorithm 1 is a useful practical aid. The paper also reports a broad experimental comparison (3 datasets, 7 baselines, 3 buffer sizes). If the mechanistic claims were supported by solid statistical evidence, the class-conditional capacity allocation would be a noteworthy contribution. However, the current evidence base is not sufficient: most accuracy gains are within one standard deviation, the ablation shows the headline gains come from the auxiliary branch plus distillation rather than from the novel routing and contrastive loss, and key hyperparameters and update rules are unspecified. The paper's contribution is therefore promising but not yet established.","major_comments":[{"comment":"The ablation in Table 3 does not support the attribution of gains to the proposed mixture-of-discretization routing. The 'branch' (plain SS2D plus ETF-KL distillation) accounts for 3.28 of the 4.24-point improvement over OnPro (33.33→36.61 vs 33.33→37.57). Adding class-conditional routing alone gives +0.31, adding LCont alone gives +0.32, and adding both gives +0.96; the reported standard deviations are 0.72–1.04, so these marginal improvements are within noise. No paired significance tests are provided. The abstract credits 'the mixture of discretization' and 'class-conditional routing' for the performance gains; this claim is currently unsupported by the reported experiments.","section":"Table 3 / Sec. 5.3"},{"comment":"The class-conditional routing rests on the uncertainty signal sigma_k, but the paper leaves its computation underspecified. The feature prototypes M_c are described as 'moving average' without giving the momentum or the update schedule. The temperature lambda_0 in Eq. (9) is a free hyperparameter and its value is not reported in Table 5 or anywhere else, although it directly controls N_k = ceil(N * sigma_k) and therefore the capacity allocated per class. Without an analysis of sensitivity to lambda_0 and a precise prototype update rule, the claim of 'dynamic, uncertainty-based adjustment' cannot be independently reproduced or tested.","section":"Sec. 4.3, Eqs. (9)–(10)"},{"comment":"The motivating premise that 'a single discretization mechanism may fail' for OCL is asserted without direct evidence, and the paper's own ablation runs against it. Replacing the proposed S6MOD core with a plain SS2D branch yields most of the improvement (36.61 vs 37.57 final accuracy), so the benefit of the mixture of discretization over a single discretization pattern is not demonstrated. A direct comparison (single discretization vs. mixture, with identical routing and losses otherwise) is needed to establish the central mechanism.","section":"Sec. 1 / Sec. 4.2"}],"minor_comments":[{"comment":"The text 'The institution is that classes' should read 'The intuition is that classes'.","section":"Sec. 4.3"},{"comment":"Table 5 lists hyperparameters only for ER with S6MOD; the statement that 'same hyperparameter settings' were applied to all baselines needs a complete listing for every baseline in the supplementary material.","section":"Table 5 / Supp. A.1"},{"comment":"The claim that 'our module is also highly effective in reducing model forgetting' is contradicted by several rows in Table 2 (e.g., MOSE on CIFAR-100 M=2000 and M=5000; MOE-MOSE on CIFAR-100 M=2000 and M=5000). The post-hoc explanation that higher forgetting is 'often due to the stronger learning capabilities' is not substantiated; please provide a quantitative analysis or soften the claim.","section":"Sec. 5.2 / Table 2"},{"comment":"The t-SNE visualizations are qualitative; adding quantitative metrics of cluster quality (e.g., silhouette score or nearest-class accuracy) would strengthen the claim of more discriminative features.","section":"Fig. 2 / Fig. 5"},{"comment":"The text says dataset settings are 'the same as those in CCLDC[45]'; reference [45] is OnPro, while CCLDC is reference [43]. Please fix the citation.","section":"References / Sec. 5.1"},{"comment":"The contrastive discretization loss uses cosine similarity without a temperature or margin parameter; please clarify the design choice and its influence on the results, especially given the small batch size.","section":"Eq. (11)"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is worth pursuing, but the experimental validation needs substantial strengthening. The ablation attribution problem and the unspecified prototype update / lambda_0 sensitivity are load-bearing and should be addressed before the paper is considered for acceptance. If the authors can provide paired significance tests and a proper single-vs-mixture discretization comparison, the contribution could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read on S6MOD. The paper does a genuinely useful thing: it shows that sticking an auxiliary SSM branch with an ETF classifier and KL distillation onto an OCL baseline improves accuracy on three benchmarks. That result is plausible, and the breadth across seven baselines is real work. The combination of mixture of discretization with class-conditional routing is new in the literature, and the math is standard selective SSM machinery with a weighted sum of expert projections. The pseudo-code matches the equations, which is more than many papers manage.\n\nThe soft spots, though, are the ones the reader flagged, and the stress-test note carries the day. The paper's own ablation (Table 3) shows that the 'branch' alone, i.e., plain SS2D plus distillation, accounts for 3.28 of the 4.24-point gain on CIFAR-100 M=2k. Adding the class-conditional routing gives +0.31, adding the contrastive loss gives +0.32, and both together give +0.96. Those marginal gains are all within one standard deviation. So the load-bearing claim that the mixture-of-discretization and routing drive the success is not supported by the evidence in the paper. The simpler explanation, a generic auxiliary branch with distillation improves OCL, is consistent with every number reported.\n\nThe uncertainty signal for routing also has a circular feel: prototypes are computed in the same feature space the branch is training, so the router is reading the model's own evolving geometry. That could be fine, but it is not anchored to an external criterion, and the sensitivity to lambda_0 is not reported. Hyperparameters are tuned per setting, no code is provided, and the SOTA claim overreaches given there is no comparison outside the seven chosen baselines.\n\nStill, the paper deserves a serious referee. The plug-and-play idea is clean, the experiments are repeated over 10 runs, and the limitations are addressable. A responsible revision would add paired significance tests, report hyperparameter selection honestly, disclose the overlap with Mamba-FSCIL, and reframe the contribution as the branch-plus-distillation with the routing as a minor addition. I would not cite it in the next year, and I'd keep it out of the reading group unless someone is specifically working on OCL. Verdict: conditional accept, but the mechanism story needs to be rewritten.","headline":"The plug-and-play SSM branch helps, but the paper's own ablation attributes the gain to the distillation branch, not the novel routing and contrastive loss.","tokens_in":18305,"tokens_out":2092,"would_cite":false,"duration_ms":18731,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that online continual learning suffers from limited adaptability, not just forgetting, and proposes a plug-and-play selective-state-space branch with class-conditional mixture of discretization that it claims improves…","keywords":["online continual learning","selective state space model","mixture of discretization","class-conditional routing","catastrophic forgetting","replay buffer","contrastive loss","Mamba"],"falsifier":"On CIFAR-100, compare the reported routing against an oracle that sets $N_k$ from true per-class test error, and against prototypes frozen after the first task. If oracle routing does not beat prototype routing, or if frozen prototypes match flexible ones, the prototype-margin uncertainty is not carrying the reported gains. A simpler check: record $\\sigma_k$ during the last task and compare it with actual per-class accuracy, since the claim predicts that high-$\\sigma_k$ classes are the misclassified ones.","tokens_in":17189,"feed_emoji":"📈","tokens_out":10620,"duration_ms":92820,"temperature":0.7,"pith_summary":"Online continual learning systems see every training example once and must keep old knowledge while absorbing new tasks. Most methods rely on replay and focus on remembering, rather than on making the model adapt quickly to the stream. This paper argues that adaptability is the missing ingredient and proposes S6MOD, a plug-and-play branch that can be attached after the backbone of an existing method. The branch runs features through a selective state space model whose discretization step is a mixture of candidate patterns, with the number of patterns chosen per class by a routing rule based on prototype margins, trained with a contrastive loss. The paper reports that S6MOD improves average accuracy in every tested configuration and lowers average forgetting in most, across seven replay baselines on CIFAR-10, CIFAR-100, and Tiny-ImageNet.","feed_headline":"One add-on branch lifts online continual learning accuracy","feed_subtitle":"Class-conditioned discretization in a selective state-space model improves seven replay baselines on three benchmarks.","key_machinery":"The load-bearing object is S6MOD, an extra branch appended after the backbone and supervised by a fixed equiangular tight frame (ETF) classifier. Inside the branch, a selective state space model with four-direction scans produces $\\hat{X} = \\mathrm{SiLU}(\\mathrm{Conv}(X))$, and a sparse mixture of discretization forms $\\Delta = \\sum_{i \\in \\Omega(\\hat{X})} w_i \\Delta_i$ from the top-$N_k$ of $N$ linear projection experts. The class-conditional router sets $N_k$ from prototype-margin uncertainty (Eqs. 9 and 10), the contrastive discretization loss (Eq. 11) enforces within-class consistency and between-class diversity of $\\Delta$, and a KL term transfers the branch predictions back to the base method head. This mechanism carries the claimed adaptability because $\\Delta$ controls the decay rate of the state update, so the selected patterns determine how much each class's features are allowed to shift.","core_discovery":"The central claim is that the bottleneck in online continual learning is model adaptability rather than memory retention alone, and that a selectively discretized state space branch restores it. S6MOD computes a per-input discretization $\\Delta$ as a weighted sum of the top-$N_k$ of $N$ learned discretization projections, with $N_k = \\lceil N \\sigma_k \\rceil$ set by the class-conditional routing rule $\\sigma_k = \\mathrm{Avg}_c\\{\\exp(-\\lambda_0 \\|M_k - M_c\\|_2)\\}$ over moving-average class prototypes. Classes whose prototypes sit close to others receive more discretization patterns, allocating extra capacity to uncertain classes, while confident classes keep fewer patterns to protect old knowledge. A contrastive discretization loss pulls the $\\Delta$ values of same-class samples together and pushes different-class samples apart, and an equiangular tight frame classifier with a KL-divergence term supervises the branch and the base head. Across seven replay baselines, the paper reports higher average accuracy and, in most settings, lower average forgetting, with the largest gains on Tiny-ImageNet and on the OnPro and MOE-MOSE combinations.","pith_inferences":["As an extension the paper does not test, the same branch could be attached to non-replay online continual learning methods or to task-agnostic streams, since it modifies feature adaptation rather than buffer management.","The prototype-margin uncertainty is one possible routing signal; replacing $\\sigma_k$ with prediction entropy or gradient-based uncertainty and checking whether results move would show whether the specific margin statistic is essential.","Because each discretization expert is a linear projection, scaling $N$ is cheap, so a natural extension is per-layer or per-scan routing instead of a single class-level $N_k$.","An ablation that isolates the dynamic $\\Delta$ from the auxiliary contrastive signal would clarify which component produces the reported gain."],"forward_implications":["Adding S6MOD to a replay-based baseline improves average accuracy in every dataset and memory-buffer configuration reported in Table 1, with gains around one percentage point on CIFAR-10 and CIFAR-100 and larger gains on several Tiny-ImageNet settings.","The routing gives new-task learning comparable to activating all $N$ patterns while keeping forgetting close to the single-pattern setting, because uncertain classes receive more discretization capacity and confident classes fewer.","S6MOD remains effective on distillation-based baselines and with different classifiers: the branch features transfer to NCM classification as well as logit classifiers, and gains persist on MOSE and MOE-MOSE.","Feature embeddings of buffered data become more compact within classes and more separated across classes after adding the branch, as shown by t-SNE visualizations.","The total pattern count $N$ controls a plasticity-forgetting trade-off, and class-conditional routing shifts that trade-off toward lower forgetting for a given $N$."],"supporting_citations":[{"why":"It supplies the selective state space (S6) mechanism whose discretization step S6MOD replaces with a mixture.","marker":"[13]"},{"why":"It supplies the four-direction SS2D scan used as the branch core in the implementation and ablation.","marker":"[26]"},{"why":"It defines the experience-replay baseline (ER) that S6MOD is first attached to.","marker":"[33]"},{"why":"It defines the OCM baseline and the mutual-information replay objective that S6MOD augments.","marker":"[17]"},{"why":"It defines the OnPro baseline and the online prototype learning setting used in comparisons.","marker":"[45]"},{"why":"It defines the OCM-CCLDC and OnPro-CCLDC baselines and the implementation protocol the experiments follow.","marker":"[43]"},{"why":"It defines the MOSE and MOE-MOSE baselines that S6MOD is tested against and attached to.","marker":"[49]"},{"why":"It provides the prior evidence that selective state space models give dynamic adaptation in few-shot class-incremental learning, which S6MOD extends to online continual learning.","marker":"[23]"},{"why":"It supplies the equiangular tight frame classifier that supervises the S6MOD branch output.","marker":"[50, 51]"},{"why":"It supplies the load-balancing loss that stabilizes the mixture-of-discretization experts.","marker":"[57]"}],"fun_headline_variants":["Plug-and-play S6MOD boosts continual learning via adaptive discretization","Class-conditional discretization sharpens online continual learning","Selective state-space branch lifts online continual learning accuracy","Adaptive discretization module enhances continual learning performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The routing's uncertainty estimate assumes that pairwise distances between moving-average class prototypes are trustworthy even though every image is seen once and old classes reappear only through a small replay buffer; if those prototypes are noisy, the number of discretization patterns $N_k$ is misallocated and the claimed stability-plasticity balance does not hold.","fun_headline_variants_meta":{"raw":{"variants":["Plug-and-play S6MOD boosts continual learning via adaptive discretization","Class-conditional discretization sharpens online continual learning","Selective state-space branch lifts online continual learning accuracy","Adaptive discretization module enhances continual learning performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000704,"raw_usage":{"total_tokens":3192,"prompt_tokens":977,"completion_tokens":2215,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":2150}},"tokens_in":593,"tokens_out":2215,"duration_ms":15364,"temperature":1.0,"reasoning_tokens":2150,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:58:40.154355+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On CIFAR-100, compare the reported routing against an oracle that sets $N_k$ from true per-class test error, and against prototypes frozen after the first task. If oracle routing does not beat prototype routing, or if frozen prototypes match flexible ones, the prototype-margin uncertainty is not carrying the reported gains. A simpler check: record $\\sigma_k$ during the last task and compare it with actual per-class accuracy, since the claim predicts that high-$\\sigma_k$ classes are the misclassified ones.","supporting_citations":[{"cited_title":"Experience replay for continual learning","cited_arxiv_id":null,"evidence_quote":"It defines the experience-replay baseline (ER) that S6MOD is first attached to."},{"cited_title":"Online contin- ual learning through mutual information maximization","cited_arxiv_id":null,"evidence_quote":"It defines the OCM baseline and the mutual-information replay objective that S6MOD augments."},{"cited_title":"Online prototype learning for online con- tinual learning","cited_arxiv_id":null,"evidence_quote":"It defines the OnPro baseline and the online prototype learning setting used in comparisons."},{"cited_title":"Improving plasticity in online continual learning via collaborative learning","cited_arxiv_id":null,"evidence_quote":"It defines the OCM-CCLDC and OnPro-CCLDC baselines and the implementation protocol the experiments follow."},{"cited_title":"Orchestrate latent expertise: Advancing online contin- ual learning with multi-level supervision and reverse self- distillation","cited_arxiv_id":null,"evidence_quote":"It defines the MOSE and MOE-MOSE baselines that S6MOD is tested against and attached to."}],"review_version":1}