{"id":"2c2c0a57-5a54-415a-bfe0-388df81d38ed","arxiv_id":"2607.25870","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"kiloVAD, a 2.1k-parameter causal CNN VAD on standard Mel features, reaches 0.850 AUC on AVA-Speech and beats standard QAT by 1–4% at INT4 via angle-based self-distillation.","lead":"A new voice-activity detector uses only standard CNN operations and Mel features, hitting 0.850 AUC at just 2,100 parameters and 200 ms latency on a tiny model for edge devices. It also introduces a quantization-training trick that recovers up to ~4% of the accuracy lost when weights are shrunk to 4-bit.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pruning-config transfer is not robust across seeds; headline 0.850 AUC rests on n=8 after 2/10 collapse, so the central claim is conditional on successful training.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the reliability of the 2.1k-parameter configuration after the 2/10 seed collapse. This is the most critical issue because the central empirical claim (0.850 AUC) is an average over only the successful runs. The paper discloses this limitation in Section 5, but the abstract and conclusion do not condition the headline on successful training. A secondary concern is the abstract's 'new state of the art' claim, which is not supported by the headline 0.850 AUC alone (AtomicVAD reports 0.869 causal in Table 1), but this is an interpretation issue that could be resolved by defining 'deployment-ready' more explicitly. The pruning robustness issue is more fundamental: if the transfer protocol is not stable across seeds, the architecture's claimed advantage over MarbleNet is itself conditional. The proposed concrete test—retraining over 10 fresh seeds and reporting the full distribution including collapsed runs—would settle whether the collapse is a rare, avoidable event or a sign of edge-of-stability behavior. The reader's CONDITIONAL verdict is appropriate; no change is needed.","tokens_in":8399,"tokens_out":10191,"duration_ms":91822,"concrete_test":"Retrain the 2.1k configuration using the exact Optuna-derived per-layer ratios on 10 fresh seeds and report the complete per-seed AUC distribution, including collapsed runs (e.g., labeling runs whose validation AUC after fine-tuning is below 0.6 as collapsed). Compute both the successful-run mean and an 'all-runs' mean that assigns collapsed runs AUC=0.5. If the successful-run mean remains ~0.850 and the collapse rate is ≤2/10, the concern is mitigated; if the all-runs mean drops below ~0.80 or the collapse rate increases, the headline claim should be re-scoped as conditional on successful training and the abstract should state the failure rate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (0.850 AUC with 2.1k parameters) is reported as the mean over 8 surviving training runs. Section 5 states that the per-layer pruning ratios were optimized on a single seed and transferred to 10 models, of which 2 collapsed due to layer collapse. The Table 1 confidence interval (0.850±0.007, n=8) excludes these failures, so it estimates performance conditional on successful training rather than the expected outcome of the published training recipe. If the 2/10 collapse rate reflects a configuration at the edge of stability rather than a rare, detectable failure, the headline result is not a robust system claim. The paper discloses this, which is commendable, but the abstract's 'achieving 0.850 AUC' and 'new state of the art' do not carry the caveat. A reader deploying this on a single training run has a 20% chance of not obtaining the reported model. Additionally, the QAT results in Figure 4 appear to come from a single model without error bars, but the pruning transfer is the more fundamental issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"kiloVAD is a CNN-only voice activity detector designed for edge deployment on standard Mel spectrogram features with a 200 ms input context. The paper contributes three main elements: (1) a per-layer structured pruning strategy with a multi-objective Optuna search and self-distillation from the unpruned model, (2) an angle-aware self-distilling quantization-aware training (QAT) method that freezes full-precision classifier weight vectors as angular prototypes, and (3) an evaluation protocol that scores each frame independently under strictly causal conditions on AVA-Speech. The central empirical claims are that the pruned 2.1 k-parameter model reaches 0.850 AUC (F1 0.783), matching MarbleNet at 43× fewer parameters and 3× lower latency; the full 81 k model reaches 0.862 AUC, and with 360 ms context reaches 0.872 AUC; INT8 post-training quantization is lossless; and angle-aware QAT improves INT4 AUC over standard STE-based QAT by 1–4% (0.811 vs. 0.800 for the 10 k model and 0.719 vs. 0.693 for the 2.1 k model). The authors emphasize deployability: no learnable filterbanks, no recurrent layers, no non-standard activations, and compatibility with TensorFlow Lite for Microcontrollers. Code, pretrained weights, and a web demo are promised.","tokens_in":8738,"tokens_out":5127,"duration_ms":50280,"significance":"If the results are robust, this is a practically valuable contribution: it demonstrates that a standard-Mel, CNN-only, TFLM-compatible model can match the accuracy of much larger or architecturally specialized compact VADs, and the compression pipeline (multi-objective per-layer pruning plus self-distillation plus angle-based QAT) is reusable for other tiny audio models. The paper's strengths include the explicit choice of strictly causal per-frame evaluation to avoid the inflated numbers produced by non-causal sliding-window protocols, a clear comparison table that separates causal from non-causal results, and a commendable disclosure that 2 of 10 runs collapsed when the pruning configuration was transferred. However, the central 0.850 AUC headline is an n=8 conditional estimate, and the QAT comparison appears to rest on single runs; both issues need to be resolved before the claims as written can be accepted.","major_comments":[{"comment":"The headline result 0.850±0.007 is computed over n=8 runs because 2 of 10 independent training runs exhibited layer collapse after transferring pruning ratios that were optimized on a single seed. This makes the central claim an estimate of performance conditional on successful training, not the expected outcome of the published recipe. A practitioner has a 20% chance of not obtaining the reported model on a single run. The paper discloses this, which is good, but the abstract and Table 1 do not carry the caveat. Please report all-seed statistics with collapsed runs scored as failures (or a clear deterministic mitigation that avoids collapse), give the failure rate in the abstract, and report the selected per-layer pruning ratios so the transfer protocol is reproducible.","section":"§5 'Structured pruning' and Table 1"},{"comment":"The QAT comparison (0.811 vs. 0.800 for 10 k; 0.719 vs. 0.693 for 2.1 k) appears to be based on a single training run for each configuration. No confidence intervals or significance tests are provided. The claimed 1–4% relative improvement is a central method contribution, and its magnitude is comparable to the seed-to-seed variability seen elsewhere in the paper (e.g., ±0.007 AUC for the pruned models). Please provide multi-seed results with error bars, or explicitly state that the improvement was observed in a single run and temper the claim accordingly.","section":"§5 'Quantization' and Figure 4"},{"comment":"The abstract states that the 0.850 AUC result with 2.1 k parameters establishes 'a new state of the art for causal, deployment-ready VAD,' but Table 1 shows AtomicVAD's causal AUC of 0.869, and the paper's own 0.872 AUC result is for the 81 k full model at 360 ms context, not for the 2.1 k model. If 'deployment-ready' is intended to exclude AtomicVAD because of its GGCU activation or raw-audio frontend, that comparison class must be defined explicitly and the claim should be aligned with the actual 0.872 result or relaxed to, e.g., 'state of the art among standard-operator causal VADs.' As written, the abstract's claim is not supported by the table.","section":"Abstract and §5 'Comparison to prior work'"},{"comment":"The paper does not report the actual per-layer pruning ratios selected by the Optuna search, the repel weight λ in Eq. (1), or the soft-to-hard annealing schedule used for the reported QAT results. These are free parameters tuned on validation data and are necessary for reproducibility of the 0.850 and 0.872 results. Please include them in a table or the supplementary material.","section":"§3.1–3.2 and §5"}],"minor_comments":[{"comment":"The caption says 'open vs. filled markers indicate 2.1 k vs. 10 k parameters,' but the legend labels methods. In grayscale printing the marker shape is the only cue; please add an explicit legend entry for parameter count.","section":"Figure 4"},{"comment":"R1–R4 are defined in the header but not in the caption. Add a caption sentence such as 'R1: standard Mel frontend; R2: portable ops; R3: low latency; R4: causal evaluation.'","section":"Table 1"},{"comment":"The text says normalization is computed over '21 time steps for 64 mel bins.' Please specify the frame shift/hop length and window size so a reader can reproduce the 200 ms context yielding 21 time steps.","section":"§2.1"},{"comment":"The 1–4% improvement is reported as 'relative'; state this explicitly in the text and in the abstract to avoid confusion with absolute AUC differences.","section":"§5 'Quantization'"},{"comment":"The hinge-style repulsion term has no margin parameter; it activates as soon as any non-target cosine similarity exceeds zero. This may be intentional, but the choice of zero margin and any sensitivity to it should be mentioned in the text.","section":"Eq. (2)"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid empirical core and the authors are unusually transparent about the pruning-transfer failure, but the n=8 issue is central to the headline claim. I would not accept the paper in its current form with the abstract's unconditional '0.850 AUC' and 'state of the art' language. The requested revisions—all-seed statistics, multi-seed QAT comparisons, explicit comparison-class definition, and reporting of hyperparameters—are within the paper's scope and should be achievable. Once these are addressed, the contribution is likely suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — this is a genuine, useful tiny-VAD paper that mostly lives up to its abstract, with one important asterisk. What's new is not the architecture — a CNN-only stack on Mel features is a sensible synthesis of known blocks — but two concrete engineering contributions: per-layer structured pruning tuned via Optuna with self-distillation, which gets to 2.1k params while keeping 0.850 AUC, and an angle-based self-distilling QAT objective that freezes the full-precision classifier and pulls penultimate features toward target prototypes. That loss is a real novelty, not just CosFace re-applied, and it buys 1–4% over standard STE QAT at INT4. The causal evaluation protocol is also a point of integrity: 200ms context, no future frames, no smoothing, and they report confidence intervals for the main results.\n\nCredit where due: they disclose that pruning ratios were optimized on a single seed and then transferred to 10 fresh models, and that 2 of those collapsed — so the headline 0.850±0.007 is n=8, conditional on successful training. That is exactly the kind of transparency we want. The 360ms result (0.872) beats AtomicVAD's causal 0.869, but that's with 360ms, not the 200ms headline, and the abstract's 'new state of the art' is too strong given the caveats.\n\nSoft spots: The abstract hides the n=8 issue; a reader deploying this on a fresh training run has a 20% chance of not getting the advertised model. That's a real robustness concern, and the paper should say so in the abstract or at least temper the SOTA claim. Figure 4, the QAT comparison, appears to be from one model per configuration with no error bars, so the 1–4% gain could be within run-to-run noise. The per-layer pruning search also introduces hidden degrees of freedom — the ratios are not reported, so others can't reproduce the exact configuration without guessing. That's not fatal, but it limits the 'reusable' claim.\n\nAlso, the comparison table mixes protocols: TinyVAD and SincQDR use non-causal sliding windows, and the paper notes this, but the table's row-by-row checkmarks make it easy to misread equal footing. A more careful presentation would flag AUCs that are not directly comparable.\n\nBottom line: this paper deserves a serious referee. The empirical claims are plausible, the method is novel enough, and the authors are transparent about failure modes. The fixes are incremental — add error bars to Figure 4, report pruning ratios, and re-scope the abstract. I'd engage with it in the review process, not desk-reject.","headline":"A solid tiny-VAD engineering paper with a genuinely new QAT loss and honest failure reporting; just don't read the abstract's SOTA claim without the n=8 caveat.","tokens_in":9198,"tokens_out":2469,"would_cite":true,"duration_ms":20946,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 2.1-thousand-parameter VAD matches MarbleNet's accuracy at 43x fewer parameters under strictly causal, 200 ms evaluation.","keywords":["voice activity detection","edge deployment","structured pruning","quantization-aware training","self-distillation","convolutional neural networks","Mel spectrogram","low-latency inference"],"falsifier":"Retrain the 2.1k-parameter configuration on, say, 30 random seeds with the same pruning ratios and protocol; if more than ~10% of runs collapse to unusable AUC, or if the mean AUC including those failures drops below 0.850, the central 'matches MarbleNet' claim would not hold as a system-level guarantee. Alternatively, run the same per-layer pruning + self-distillation pipeline on a different VAD dataset (e.g., a far-field noisy set) and check whether the 2.1k configuration stays within 1.3% of the unpruned baseline.","tokens_in":8327,"feed_emoji":"🎙️","tokens_out":6843,"duration_ms":53277,"temperature":0.7,"pith_summary":"The paper argues that a voice activity detector built strictly from standard, widely supported components—fixed Mel spectrogram input, convolutional layers only, and a 200 ms causal evaluation window—can match the accuracy of much larger models. It claims kiloVAD, at 2.1 thousand parameters, achieves 0.850 AUC on AVA-Speech, the same score as the 91-thousand-parameter MarbleNet but with 43x fewer parameters and 3x lower latency. The paper also introduces two compression techniques: per-layer structured pruning guided by multi-objective search plus self-distillation, and an angle-aware self-distilling quantization-aware training loss that improves INT4 accuracy by 1–4% over standard QAT. If these claims hold, always-on edge devices can run a VAD on microcontrollers using only operators that are already supported by embedded runtimes.","feed_headline":"2.1K-parameter VAD matches MarbleNet with 43x fewer weights","feed_subtitle":"CNN-only, standard-Mel design runs on 200 ms causal frames at 0.850 AUC, ready for always-on microcontrollers.","key_machinery":"The load-bearing mechanism is a compression pipeline: (1) per-layer structured pruning via dependency-graph analysis with layer-specific ratios found by multi-objective search (minimizing false positives at fixed true-positive rate and parameter count), followed by fine-tuning with self-distillation from the unpruned model; (2) an angle-aware self-distilling QAT loss (Eqs. 1–2) that, with a frozen full-precision classifier, pushes each penultimate feature toward its own class weight vector in cosine space and pushes it away from the other class's vector, explicitly countering the angular distortion introduced by low-bit quantization. The architecture itself—a 1x1 adapter, depthwise separable","core_discovery":"In the paper's own terms, the central discovery is that extreme compression and deployment compatibility don't have to trade away VAD accuracy when the architecture is co-designed for pruning and quantization. The design separates Mel resolution from channel width via a 1x1 adapter, uses global average pooling to decouple parameter count from input length, and normalizes each frequency bin per-frame. The resulting CNN-only model, unpruned at 81.1k parameters, scores 0.862 AUC; pruning to 2.1k parameters with per-layer ratios and self-distillation retains 0.850 AUC, within 1.3% of the baseline while cutting MACs from 1.7M to 44k. Under the same causal protocol, this matches MarbleNet's 0.850","pith_inferences":["Because the angle-aware QAT loss only requires a frozen full-precision classifier, the same training recipe could be lifted onto other tiny classification models, such as keyword spotting, where angular error dominates at low bit-widths—a testable extension the paper does not run.","The pruning-ratio transfer protocol implies a single multi-objective search per architecture might serve many deployments; if the two collapsed seeds instead reflect run-level instability, a production guardrail would be to verify any pruned configuration on multiple initializations before shipping.","The 200 ms performance plateau aligns with the 4–5 Hz syllable rate of speech, suggesting an adaptive context length could trade latency against accuracy in very noisy domains—an idea the paper leaves implicit.","A direct follow-up could test whether per-frame normalization plus pruning generalizes to far-field or multilingual speech, where the frozen teacher's class prototypes may be less stable."],"forward_implications":["A causal, 200 ms-context VAD with only 2.1k parameters can be deployed on microcontroller-class hardware using standard Mel frontends and CNN ops, a capability current compact VADs lack.","The per-layer pruning plus self-distillation recipe is claimed to generalize across compression levels and seeds (10 independent models, 8 surviving), offering a path to hit arbitrary parameter budgets.","Angle-aware QAT gives a usable INT4 regime: the 2.1k model goes from 0.693 AUC (standard QAT) to 0.719 AUC (angle-aware), and INT8 stays lossless at 0.851.","Extending context to 360 ms yields 0.872 AUC, which exceeds AtomicVAD's causal 0.869 with 630 ms context—suggesting latency can be traded for accuracy without leaving standard operators."],"fun_headline_variants":["2.1K-parameter VAD matches MarbleNet","Pruned VAD: 43x fewer weights, same AUC","CNN-only VAD for edge: 2.1K params","Causal VAD with 2.1K weights hits 0.850 AUC"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim that 2.1k parameters is a reliable operating point rests on the assumption that the two training runs that collapsed under the transferred pruning ratios are rare, avoidable failures rather than a signal that this configuration sits at the edge of a stability cliff.","fun_headline_variants_meta":{"raw":{"variants":["2.1K-parameter VAD matches MarbleNet","Pruned VAD: 43x fewer weights, same AUC","CNN-only VAD for edge: 2.1K params","Causal VAD with 2.1K weights hits 0.850 AUC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000445,"raw_usage":{"total_tokens":2066,"prompt_tokens":699,"completion_tokens":1367,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":443,"completion_tokens_details":{"reasoning_tokens":1289}},"tokens_in":443,"tokens_out":1367,"duration_ms":8988,"temperature":1.0,"reasoning_tokens":1289,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T01:14:37.715324+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the 2.1k-parameter configuration on, say, 30 random seeds with the same pruning ratios and protocol; if more than ~10% of runs collapse to unusable AUC, or if the mean AUC including those failures drops below 0.850, the central 'matches MarbleNet' claim would not hold as a system-level guarantee. Alternatively, run the same per-layer pruning + self-distillation pipeline on a different VAD dataset (e.g., a far-field noisy set) and check whether the 2.1k configuration stays within 1.3% of the unpruned baseline.","supporting_citations":[],"review_version":1}