{"id":"eff360ff-5e72-44d4-9730-24c13426e69b","arxiv_id":"2607.19918","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Freezing a deployed keyword-spotting model and training only a small attached branch with a separate head adds new keywords with no change to old outputs, cutting new-keyword false reject rate from 6.46% to 4.37% versus a same-budget separate model.","lead":"This paper freezes a deployed keyword-spotting model and trains only a small extra branch plus a new head to add keywords, keeping all original outputs exactly unchanged. In experiments on Google Speech Commands, the approach cuts average new-keyword false reject rate from 6.46% to 4.37% versus a parameter-matched separate-model baseline while using fewer computations than adapters or LoRA.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Expansion depth L is selected on the same held-out keyword pair used in the headline, so the reported FRR advantage may partly reflect tuning on the evaluation data.","rationale":"The reader's verdict is CONDITIONAL, with the weakest assumption stated as feature transferability and background representativeness, tested only by a depth sweep that selects L on the held-out pairs. My stress-test converges on a narrower, more specific version: the selection of L itself is the load-bearing weak point, because it directly affects the headline numbers. The reader's concern about out-of-sample validation is correct, but I would elevate it to the primary issue: the empirical superiority claim rests on comparing a method whose main hyperparameter was chosen after seeing the evaluation result against baselines whose hyperparameters were chosen to match compute, not accuracy. This is not an internal inconsistency and does not undermine the non-regression guarantee, which is valid by construction. The paper does have independent support: the freezing argument is parameter-free and the MAC comparison is concrete. The concern is testable with a simple nested-selection protocol. If that test passes, the central empirical claim would be substantially stronger; if it fails, the reported advantage may be an artifact of selection bias. Since the reader already assigned CONDITIONAL, no verdict change is needed; the test would either upgrade to ACCEPT or push toward REJECT depending on the outcome.","tokens_in":8438,"tokens_out":5788,"duration_ms":66303,"concrete_test":"Run a leave-one-pair-out selection experiment. For each of the five Y2 pairs p, choose L (and, for LoRA/Adapters, the corresponding rank/depth) using only the other four keyword pairs as validation, then evaluate on the left-out pair p. Report the macro-average FRR across the five left-out evaluations at the same 1% FAR on Common Voice. If the average remains clearly below the LoRA/Ensemble values and within the original confidence intervals, the selection-bias concern is refuted. If the average rises toward 6% or the advantage disappears, the headline comparison was driven by tuning L on the evaluation pairs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The core non-regression claim is sound by construction: freezing the base path and its BN statistics guarantees identity of core logits (Eq. 3). The load-bearing part is the empirical claim that the proposed expansion improves new-keyword FRR over Ensemble/LoRA. The weakest step in that claim is the selection of the expansion depth L. Section 4.5 and Figure 3 report the FRR-vs-L curve only for Y2 = {left, right}, and then state 'we adopt the four-block expansion as the optimal configuration for all reported comparisons.' Table 2's headline average includes that same {left, right} pair. Thus one of the five evaluation tasks contributes a result obtained after inspecting that task's own FRR curve for L, while the other four pairs are evaluated under a depth chosen on a different task. No nested or out-of-sample selection is reported. Moreover, baseline Adapters/LoRA configurations are chosen to match compute budgets, not accuracy-tuned on a validation split, so the comparison may conflate the modular-expansion mechanism with more favorable hyperparameter selection. This does not threaten the invariance guarantee, but it does threaten the central 'outperforms LoRA/Ensemble' conclusion: the 4.37% vs 6.41% margin could shrink if L were selected honestly out-of-sample.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a modular expansion method for incremental keyword spotting (KWS). The deployed base model, including batch-normalization parameters and running statistics, is frozen; a lightweight expansion branch (≤10k added parameters) taps intermediate activations of the frozen encoder and feeds a separate new-keyword head. Since the core head is not changed by the expansion path, the core logits and the core-first decision rule are preserved exactly for all inputs (Eq. 3). Training uses only new-keyword data D2. Experiments on Google Speech Commands v2 with Common Voice negatives report new-keyword and core FRR at 1% FAR, averaged over 8 seeds with 95% CIs, and compare against full fine-tuning, EWC, head-only, ensemble, adapters, and LoRA. The headline result is average new-keyword FRR of 4.37% vs 6.46% for the ensemble baseline and 6.41% for LoRA under the same added-parameter budget, with lower worst-case MACs.","tokens_in":8761,"tokens_out":6830,"duration_ms":77964,"significance":"If the empirical comparison holds, the paper has clear practical value: it provides a strict, by-construction non-regression guarantee for deployed KWS detectors while adding new keywords with a small parameter and compute overhead. Strengths include the exact preservation argument of Eq. (3), evaluation on five held-out keyword pairs, confidence intervals over 8 runs, and parameter/MAC-budget matching. However, the central empirical claim is weakened by a model-selection procedure that uses the evaluation data (the expansion depth L is chosen from FRR on the {left,right} pair and then used in the same pair's headline result), and by baseline hyperparameters that are compute-matched but not accuracy-tuned. With an honest out-of-sample selection protocol, the method would be considerably more convincing.","major_comments":[{"comment":"Expansion depth L=4 is selected from the FRR-vs-L curve computed for Y2={left,right} and then adopted for all reported comparisons. Table 2's average and the {left,right} row both include exactly this task. No validation split is used for architectural selection, so the reported 4.37% average partly reflects tuning on the evaluation data. This is selection bias rather than circularity in Eq. (3), but it is load-bearing because the paper's claim is empirical superiority. Please provide a nested or out-of-sample selection protocol (e.g., choose L on a validation split for each pair, or fix L on a separate development pair) and report results under that protocol; also report sensitivity of the Table 2 conclusions to L.","section":"§4.5, Fig. 3, and Table 2"},{"comment":"Adapters and LoRA are configured to match the added-parameter budget and inference compute, but no accuracy-based validation selection is reported for their depth/rank. If the proposed method's hyperparameter L is tuned on the test tasks (see previous comment) while the baselines are only compute-matched, the observed 2.09-point FRR gain may partly reflect a hyperparameter advantage rather than the modular-expansion mechanism. Please tune all methods under the same validation-based protocol, or state explicitly why compute matching alone is the appropriate comparison, and show the resulting performance for all methods.","section":"§3.5 and §4.2"},{"comment":"The claim that the proposed method 'outperforms' LoRA and Ensemble rests on point estimates with overlapping CIs for some individual pairs (e.g., {up,down}: 4.57±0.73 vs LoRA 5.12±1.78). The average CIs are more favorable, but with the L-selection issue above, the CI coverage is not trustworthy. After fixing the selection protocol, please also report paired or per-task comparisons and, if possible, the number of pairs for which the proposed method is statistically better.","section":"§4.4 / Table 2"}],"minor_comments":[{"comment":"The sentence 'the concatenation reduces to an identity' for the first Expanded Block is confusing: if there is no previous expanded output, the block's input is just the frozen activation, which is not an identity operation. Please clarify the exact first-block computation.","section":"§3.3"},{"comment":"The notation switches among score_y, s_core_y, and s_new_y. Please define s_core_y explicitly before Eq. (5) and keep the notation consistent throughout.","section":"§3.4, Eq. (5)"},{"comment":"The table omits core FRR for methods that freeze the core path because they match Base Model exactly; this is reasonable, but the caption should state that these values are identical by construction rather than only mentioning it in the text.","section":"Table 2"},{"comment":"The y-axis label 'FRR' should specify whether it is macro FRR for Y2 at the same 1% FAR threshold used in Table 2, and whether the curve is for the {left,right} pair only or averaged over pairs.","section":"Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The central construction is sound and the paper is potentially a good fit for the venue. The main obstacle is the test-set-based selection of L and the asymmetric hyperparameter treatment of baselines; both are fixable. I would be willing to see a revision that adds a clean validation-based model-selection protocol and re-runs the comparison under the same protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline result is that freezing the deployed KWS detector and attaching a ≤10k-parameter expansion branch gives a strict non-regression guarantee by construction, and on Google Speech Commands the branch beats parameter-matched Ensemble and LoRA baselines on new-keyword FRR. I think the guarantee is real and the empirical claim is likely true, but the paper has a selection-bias asterisk you should know about before citing the exact numbers.\n\nWhat's new: framing class-incremental expansion for a single always-on detector with frozen BN statistics, a separate new-keyword head, and a core-first decision rule. The relation to Progressive Neural Networks and LoRA/adapters is honestly stated, and the evaluation is better than typical for this area: 8 runs, 95% CIs, FAR calibrated on Common Voice negatives, and deployment-safe baselines compared under the same added-parameter budget. The invariance in Eq. 3 is correct. That is real, useful engineering.\n\nSoft spots, in proportion. First, the expansion depth L=4 is selected by inspecting Figure 3 on the {left, right} held-out pair, and that same pair is included in the headline average. So one of the five evaluation tasks is in-sample for the depth choice. That is genuine selection bias, but it is not load-bearing: on the other four pairs, the proposed method still beats LoRA by roughly 1.8 points average FRR and Ensemble by 2.1 points. If you exclude {left, right}, the conclusion holds. Still, they should do the sweep on a separate validation pair or report a nested selection.\n\nSecond, the baseline hyperparameters (LoRA rank, adapter width, L for the parallel branch) are chosen to match MACs, not accuracy-tuned on a validation split. That could stack the deck in favor of the proposed mechanism, though the margins are large enough that I doubt tuning would erase them.\n\nThird, no code or full hyperparameter details are released. For a paper whose value is primarily empirical, that is a real limitation.\n\nOverall: the central argument holds up, and the paper is a solid incremental contribution for embedded KWS. It does not pretend to be open-vocabulary; it is about fixed additions under strict update safety. Worth serious peer review, with a request for honest depth selection, better baseline tuning, and ideally code. I'd bring it to a reading group for the non-regression guarantee alone, and I'd cite it in work on safe incremental updates.\n\nRecommendation: send to peer review.","headline":"A genuinely useful by-construction non-regression guarantee, with FRR gains that mostly survive an honest re-look at the depth selection.","tokens_in":9223,"tokens_out":2814,"would_cite":true,"duration_ms":28409,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deployed keyword-spotting model can accept new keywords by freezing the core and attaching a small trainable branch, with old triggers unchanged by construction.","keywords":["keyword spotting","class-incremental learning","modular neural networks","embedded devices","non-regression guarantee","false reject rate","parameter-efficient expansion","on-device deployment"],"falsifier":"Hold out a sixth keyword pair, run the depth sweep only on the five pairs used in the paper, then evaluate on the held-out pair: if the L=4 configuration does not reproduce the ~2-point FRR gain over the ensemble baseline on the unseen pair, the reported improvement is an artifact of selecting L on the evaluation set. Alternatively, train the expansion on keywords from a language not seen by the base model and compare against the separate-model ensemble.","tokens_in":8346,"feed_emoji":"🔊","tokens_out":4297,"duration_ms":43403,"temperature":0.7,"pith_summary":"The paper tries to establish that an embedded keyword-spotting system can grow its vocabulary after deployment without any risk of regressing on existing keywords. The method freezes the entire deployed network—including batch-normalization statistics and the core classifier—and attaches a parameter-capped (≤10k) expansion branch with a separate head for new keywords, trained only on new-keyword data. Because the core computation is untouched, the original logits and thresholds are identical for every input, so the shipped decision rule for old keywords is preserved exactly. On a standard 10-command benchmark at a fixed 1% false-accept operating point, the expanded model reports average new-keyword false-reject rates of 4.37%, versus 6.46% for a parameter-matched separate-model ensemble and 6.41% for LoRA, while using fewer multiply-accumulate operations.","feed_headline":"Freeze the core, add a tiny branch to expand keywords","feed_subtitle":"New keywords drop to 4.37% FRR at 1% FAR while old triggers stay bit-for-bit unchanged.","key_machinery":"Expanded Blocks: each block taps the activation of a frozen base encoder block, concatenates it with the previous expanded block's output (identity for the first block), and applies a light residual transform—a small-kernel 1-D convolution, batch-normalization, and hard-swish. The final expanded state is pooled and fed to a separate New Head for Y2 ∪ {∅}. The carrying identity is Eq. (3): since the Core Head's computation is frozen and independent of the expansion path, the core logits are exactly the base logits for every input, giving a non-regression guarantee by construction rather than by regularization.","core_discovery":"The central discovery is that the strict safety guarantee does not cost accuracy: freezing the base and reading its intermediate activations through a lightweight branch transfers better to new keywords than retraining a separate model of the same budget, while preserving core behavior exactly. Concretely, the paper proves ℓ_exp_core(x) ≡ ℓ_base_core(x) for all x, meaning the probability the deployed detector assigns to any original keyword or the background class is unchanged for every input. The branch is trained solely on new-keyword data with cross-entropy, and the inference rule first runs the core detector and only consults the new head when the core rejects. Empirically this configura","pith_inferences":["If the frozen features do not transfer to the new vocabulary—say, new keywords from a language or acoustic domain absent from base training—the 4.37% figure should deteriorate toward or below the separate-model baseline; that is a testable boundary of the method.","The depth hyperparameter L is selected on the same held-out keyword pairs used to report the headline numbers, so the gain may be optimistically biased; evaluating on a further held-out pair would tighten the claim.","The core-first decision rule means new-head false accepts can only occur on utterances the core rejects; tuning the two thresholds jointly, rather than at independent 1% FAR, could shift the operating curve.","The same modular expansion recipe—freeze, attach a small branch, separate head—may apply to other class-incremental problems on embedded models where the base features are rich enough."],"forward_implications":["Existing keyword detections are bit-for-bit unchanged after expansion, so the shipped user experience cannot regress on old triggers.","New keywords can be added using only recordings of those keywords plus a background set; the original training data need not be stored or reused.","Under the same ≤10k added-parameter budget, the expansion branch outperforms separate-model, adapter, and LoRA baselines on new-keyword detection at a fixed FAR.","Worst-case inference cost is 16.34M MACs, lower than the 18.45M/20.52M of adapter/LoRA alternatives, under the core-first decision rule.","The expansion works best when tapping four of the base blocks; deeper layers are too specialized to the core vocabulary to help new keywords."],"fun_headline_variants":["Freeze core, add branch: keywords expand without regression","Freeze core, branch out: new keywords, zero regression","Add keywords without retraining: freeze core, expand branch","From 6.46% to 4.37% FRR: modular expansion freezes core","Tiny branch, exact core: new keywords without regressions"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole gain rests on the premise that a ~10k-parameter branch reading only frozen base features can learn reliable new-keyword boundaries and a shared background from new-keyword data alone; if the frozen features are not transferable, or if the new head's negative distribution at deployment differs from its training background, the reported FRR advantage collapses.","fun_headline_variants_meta":{"raw":{"variants":["Freeze core, add branch: keywords expand without regression","Freeze core, branch out: new keywords, zero regression","Add keywords without retraining: freeze core, expand branch","From 6.46% to 4.37% FRR: modular expansion freezes core","Tiny branch, exact core: new keywords without regressions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000607,"raw_usage":{"total_tokens":2635,"prompt_tokens":686,"completion_tokens":1949,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":1857}},"tokens_in":430,"tokens_out":1949,"duration_ms":12164,"temperature":1.0,"reasoning_tokens":1857,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T11:17:43.552450+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out a sixth keyword pair, run the depth sweep only on the five pairs used in the paper, then evaluate on the held-out pair: if the L=4 configuration does not reproduce the ~2-point FRR gain over the ensemble baseline on the unseen pair, the reported improvement is an artifact of selecting L on the evaluation set. Alternatively, train the expansion on keywords from a language not seen by the base model and compare against the separate-model ensemble.","supporting_citations":[],"review_version":1}