{"id":"fb0261d1-14c5-4929-b842-1e41cd1c4d89","arxiv_id":"2509.01087","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A tri-stage training scheme with a lightweight noise-disentanglement module improves Conformer-Transducer ASR robustness on noisy speech by up to 25.7% relative WER without degrading clean speech performance.","lead":"This paper introduces a noise-cleaning module for end-to-end speech recognition that strips noise from internal speech representations before decoding. It reports up to 25.7% relative word error rate reductions on noisy test sets with almost no added parameters.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing matched no-NoisyD tri-stage control at 360h: headline gains may partly reflect training schedule, not the NoisyD module.","rationale":"The paper is a coherent empirical study with useful ablations at the 100-hour scale: Table 3 shows that the full NoisyD module (S3) outperforms the simplified Encoder-C-only variant (S4), and that the tri-stage schedule alone (B3+Tri in Table 2) does not produce gains. This is genuine supporting evidence that the module matters in that setting. However, the strongest quantitative claim in the abstract is based on the 360-hour experiments (Table 5), where no equivalent no-NoisyD tri-stage baseline is provided. Since B3+ and S3+ differ in both architecture and training protocol, the 25.7% relative improvement cannot be unambiguously attributed to the NoisyD module without that control. This concern does not undermine the paper's basic plausibility; it means the central claim should remain conditional on an additional baseline. I therefore keep the reader's CONDITIONAL verdict rather than moving to ACCEPT or REJECT. The proposed test is a single, feasible training run that would settle whether the concern lands.","tokens_in":15986,"tokens_out":5835,"duration_ms":69388,"concrete_test":"Run a B3+Tri-360 baseline: use the same B3+ Conformer-Transducer architecture and the same 360-hour clean + synthetic-noisy training data, and apply the same three-stage schedule as S3+ but without the NoisyD module (pretrain on train-clean-360, then fine-tune on clean+noisy for the same number of steps as NoisyD training and fine-tuning stages). Evaluate on the Table 5 simulated noisy test sets and clean test sets, ideally with at least three random seeds. If B3+Tri-360 matches S3+ within about 1 WER on the noisy sets, the NoisyD module contributes little at 360 hours and the headline 25.7% relative gain is largely a training-schedule artifact; if S3+ retains a clear margin, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result (Table 5: up to 25.7% relative WER reduction at SNR=15, and clean test-other 16.7→14.3) compares S3+ (NoisyD + tri-stage) against B3+, a Conformer-Transducer trained from scratch on combined clean+noisy data. The two systems differ in both the architecture and the training schedule: S3+ first pretrains on clean-only data, then runs noisy-disentanglement training, then fine-tunes. The only control that isolates the tri-stage schedule from the NoisyD module is B3+Tri, reported only for the 100-hour setting (Table 2), where tri-stage alone is not beneficial (avg 22.0% vs 21.5%). But training-schedule effects can interact with dataset size and noise diversity; the 360-hour experiments, which produce the headline numbers, have no B3+Tri-360 control. Consequently, the claim that the NoisyD module itself is responsible for the large 360-hour gains is not fully isolated. The reader's concern about MSE alignment in the clean representation space is real, but it is partially addressed by the S4 ablation (Encoder-C + L_CON alone still helps). The missing matched control is more load-bearing because it directly tests whether the central architectural contribution is necessary at the scale where the strongest claim is made. Additionally, the CHiME-4 real-noise differences are small (e.g., BUS 39.6→38.3) and no confidence intervals or repeated-seed results are provided, so some reported improvements may be within run-to-run variability.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes NoisyD-CT, a Conformer-Transducer ASR system augmented with a lightweight NoisyD module inserted between the Conformer encoder and the RNN-T decoder. NoisyD consists of Encoder-C, Encoder-N, and Decoder-CN. Two auxiliary losses are added: a clean-representation consistency loss L_CON that aligns Encoder-C's output with representations from a frozen clean-only pretrained encoder, and a noisy reconstruction loss L_R that requires the concatenation of Encoder-C and Encoder-N outputs to reconstruct the noisy encoder representation. Training uses a tri-stage schedule: clean pretraining, noisy-disentanglement training with the backbone frozen, and fine-tuning of the backbone. Experiments on LibriSpeech (100h and 360h) with simulated CHiME-4 noise, real CHiME-4 test sets, and unseen DEMAND noise show consistent WER gains over Conformer-Transducer baselines, with headline relative reductions of 25.7% on simulated noisy speech and 10.6% on CHiME-4, plus small clean-test improvements.","tokens_in":16388,"tokens_out":8225,"duration_ms":91953,"significance":"The contribution is practical and the experimental design is above the minimum bar: the paper includes a schedule-only control (B3+Tri) at 100h, ablations that separate L_CON from the full reconstruction path (S4 vs S3), matched/unseen noise conditions, and a modest parameter overhead (1.71M). If the results are reproducible, NoisyD-CT is a useful add-on for noise-robust E2E ASR without external enhancement front-ends. The main weaknesses are statistical (single runs, no intervals) and the absence of a schedule-only control at the 360h scale where the strongest claims are made. The 'disentanglement' interpretation also goes beyond what the losses formally enforce. These are fixable within the paper's scope, so I do not view them as fatal.","major_comments":[{"comment":"The 360-hour experiments compare S3+ (NoisyD + tri-stage) with B3+ (Conformer-T trained from scratch on combined clean+noisy data). These systems differ in both architecture and training schedule. The only control that isolates the schedule, B3+Tri, is reported at 100h (Table 2) and actually performs slightly worse than B3. Because schedule effects can interact with dataset size and noise diversity, the headline 360h gains (e.g., 25.7% relative at SNR=15) are not fully attributable to the NoisyD module. Please run B3+Tri on the 360h condition (or otherwise control for schedule at that scale) and report it alongside Table 5.","section":"Section 6.4, Table 5"},{"comment":"All WER results are single runs with no confidence intervals or significance tests. Several key differences are small (test-clean 7.9 to 7.8 in Table 5; CHiME-4 BUS 39.6 to 38.3 in Table 6) and could lie within run-to-run variability. Please provide repeated-seed means and standard deviations, or a significance test, for the main B3+ vs S3+ comparisons and the CHiME-4 results.","section":"Section 6, Tables 3-7"},{"comment":"The claims that Encoder-N extracts 'pure noise' and that the module performs 'disentanglement' are stronger than the losses enforce. L_R only requires that the concatenation [tilde h_clean; tilde h_noisy] reconstructs ht-noisy; it does not by itself prevent Encoder-N from encoding residual speech information or prevent redundancy between the two branches. Please add an explicit disentangling constraint (e.g., orthogonality or information-reduction) or soften the interpretation, and ideally add an analysis showing that Encoder-N captures noise rather than content (e.g., an ablation that replaces or removes Encoder-N output).","section":"Section 4.2, Eqs. (7)-(8)"}],"minor_comments":[{"comment":"The formulas are written as if L_CON and L_R were simple differences; the text says MSE. Please write the squared L2 norm explicitly.","section":"Eqs. (7)-(8)"},{"comment":"The checkmark notation is difficult to read, especially whether L_CON+LR are active during fine-tuning when the row shows ND frozen. Please clarify with a separate column or footnote.","section":"Table 3"},{"comment":"The visualization is partly circular because L_CON explicitly trains tilde h_clean to match ht. It is a useful sanity check but not independent evidence of disentanglement.","section":"Section 6.5, Figure 4"},{"comment":"Typos and grammar: 'a especially designed', 'mechanisim', 'with the model size remains unchanged', 'the whole model loss function'. Also, Eqs. (7)-(8) use inconsistent notation for the reconstruction target.","section":"Throughout"},{"comment":"The related work lists SE-based and SSL-based robust ASR methods, but no experimental comparison with such systems is included. A short comparison (or an explicit statement that this is out of scope) would help position the contribution.","section":"Sections 2 and 6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an applied ASR paper with moderate novelty. The empirical core is credible, but I would not accept without the 360h schedule control and repeated-run statistics. The main conceptual risk is the misuse of 'disentanglement'; if the authors add an orthogonality constraint, this would strengthen the paper. No concerns about citation practices or scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a decent engineering paper with a plausible central claim, but the headline 360-hour improvements are not cleanly attributed to the NoisyD module because the training schedule is confounded.\n\nWhat's actually new: the authors slot a compact NoisyD module (three MLPs, 1.71M params) between the Conformer encoder and the RNN-T decoder in a Conformer-Transducer. They train it with a clean-representation consistency loss (MSE to the output of a frozen clean-trained encoder) and a noisy reconstruction loss, plus a tri-stage schedule: pretrain on clean, freeze everything and train NoisyD, then fine-tune the backbone. That combination is new, and the paper does the right kind of comparison: matched parameter counts, ablations S1–S4, plus evaluation on simulated, real (CHiME-4), and unseen-noise (DEMAND) test sets. The gains are consistent, and the clean test sets don't degrade—test-other actually improves from 16.7 to 14.3 at 360h. Credit where due: this is a well-specified, reasonably thorough empirical study.\n\nThe soft spots, in rough order of importance:\n\n1. The missing 360h B3+Tri control. The 100h data point (B3+Tri vs B3) suggests tri-stage alone doesn't help, which weakens the stress-test concern somewhat, but the paper's biggest numbers come from the 360h setup where the schedule is never isolated. As it stands, the comparison of S3+ to B3+ attributes all the gain to NoisyD, and that's an over-attribution. A solid referee should demand that control before publication.\n\n2. Single runs, no confidence intervals. On CHiME-4 BUS the improvement is 39.6 to 38.3; that's small enough to be within run-to-run variation. The DEMAND and simulated-noise numbers are large enough that I doubt they'd vanish, but the paper would be stronger with at least three seeds.\n\n3. The \"disentanglement\" language oversells the evidence. What the module does is learn a masked-ish mapping via MSE to a clean encoder's output and a reconstruction objective; there's no direct measure that speech and noise are separated. The S4 ablation (Encoder-C + L_CON alone still helps) does address the reader's worry about the MSE target being harmful—fine—but it doesn't show true separation.\n\n4. The code is promised, not shipped. For a paper whose contribution is architectural, \"will be available\" is weak.\n\nVerdict: worth a serious referee. The central idea is sensible, the method is clearly specified, and the consistency of the gains across matched and unseen noise is evidence that something real is happening. It needs the 360h control, multi-seed stats, and a more measured abstract before it's publishable as-is.\n\nRecommendation: send it out, with a note to the authors to add the control and clean up the disentanglement claims.","headline":"Solid engineering paper with a plausible but not fully isolated claim; worth referee time, but the 360h results need a schedule control before they can be believed.","tokens_in":16879,"tokens_out":3017,"would_cite":true,"duration_ms":32610,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Inserting a 1.71M-parameter disentanglement module between encoder and decoder, trained with two losses under a tri-stage schedule, cuts noisy ASR word error by up to 25.7% on simulated and 10.6% on real CHiME-4 tests.","keywords":["noise-robust speech recognition","Conformer-Transducer","noisy disentanglement","clean representation consistency loss","noisy reconstruction loss","tri-stage training","RNN-T","speech enhancement"],"falsifier":"Replace the frozen clean encoder that produces the regression target with a differently initialized Conformer encoder of the same size, keep every other detail fixed, and retrain only stages two and three; if the CHiME-4 WER gains disappear, the benefit depends on the teacher representation space rather than on the disentanglement structure itself.","tokens_in":15864,"feed_emoji":"🎙️","tokens_out":7388,"duration_ms":79619,"temperature":0.7,"pith_summary":"The paper sets out to show that end-to-end speech recognition can be made noise-robust by cleaning up internal representations rather than by adding a separate speech-enhancement front-end. It inserts a lightweight module between the Conformer encoder and the Transducer decoder that tries to split each noisy representation into a clean stream and a residual noise stream, sending only the clean stream on to the decoder. Two mean-squared-error losses guide the split: one pulls the clean stream toward the representation the same encoder produces for the corresponding clean utterance, and one requires the two streams together to reconstruct the original noisy representation. A three-stage schedule first trains the recognizer on clean speech, then trains the disentanglement module alone with everything else frozen, then fine-tunes the backbone. On LibriSpeech with CHiME-4 noise, the reported result is up to 25.7% relative word-error reduction on simulated noisy test sets, 10.6% on real CHiME-4 test sets, and no loss on clean test speech.","feed_headline":"1.71M-parameter block cuts noisy ASR word errors by 25.7%","feed_subtitle":"An in-model disentangler cleans representations before the decoder, keeping clean speech accuracy intact.","key_machinery":"The load-bearing mechanism is the NoisyD module: three four-layer MLPs (Encoder-C, Encoder-N, Decoder-CN) totaling 1.71M parameters, inserted between the Conformer encoder and the Transducer decoder. Encoder-C outputs the cleaned representation that feeds the decoder; Encoder-N extracts a residual noise representation; Decoder-CN reconstructs the original noisy representation from the concatenation. The reconstruction constraint makes the clean/noise split lossless, and the clean consistency loss L_CON anchors the clean branch to a frozen clean-trained encoder's representation via MSE. The tri-stage training schedule determines when each loss is active: clean pre-training establishes the rec","core_discovery":"The central claim is that a compact module between the Conformer encoder and the RNN-T decoder can make noisy internal representations close enough to clean representations that the decoder transcribes them with far fewer errors. Encoder-C extracts a cleaned representation for the decoder; Encoder-N extracts a noise-only stream; Decoder-CN reconstructs the original noisy representation from both. Loss L_CON pulls Encoder-C's output toward the frozen clean encoder's output for the same utterance; loss L_R forces the split to be information-complete. A tri-stage schedule pre-trains Conformer-T on clean speech, trains the module alone with the backbone frozen, then fine-tunes only the backbone.","pith_inferences":["An implication the authors leave implicit: because the cleaning happens on encoder representations rather than waveforms, the same NoisyD recipe should transfer to other encoder-decoder ASR models, provided their encoders are stable enough to serve as regression targets.","The regression target comes from a frozen clean-trained encoder, so the method's ceiling is set by that teacher's representation space; upgrading the encoder later would require regenerating the paired targets and re-running stages two and three.","A cheap practical extension would generate the clean-noisy pairs on the fly with any noise-corruption pipeline, eliminating the need for a fixed paired corpus and potentially widening the SNR range covered during training.","It is an open, testable question whether representation-level MSE alignment is more ASR-friendly than waveform-level speech enhancement; a direct comparison at equal parameter count would clarify where the benefit comes from."],"forward_implications":["The added 1.71M parameters are the only permanent cost at inference: the extra pre-trained encoder copy, Encoder-N, and Decoder-CN are discarded, so the method is a cheap retrofit to existing Conformer-Transducer systems.","Clean speech is not traded off: on the 360-hour setup, test-clean WER moved from 7.9% to 7.8%, and test-other from 16.7% to 14.3%, so the module can be inserted without a clean/noisy performance trade-off.","The gains transfer to noise types never seen in training: on DEMAND-based tests the model reported relative WER reductions of 12.4% at -5 dB and 25.5% at 15 dB.","The ablation results imply that both reconstruction (Encoder-N + Decoder-CN + L_R) and the tri-stage schedule are load-bearing; removing them leaves the system close to the baseline, and the schedule alone does not help.","Because the pre-trained clean encoder provides the regression target, the method inherits that encoder's representation quality as a ceiling on how clean the disentangled representation can become."],"supporting_citations":[{"why":"Conformer encoder architecture whose frozen representations define the representation space the NoisyD module operates on.","marker":"[38]"},{"why":"Transducer formulation that, with Conformer, forms the baseline Conformer-Transducer the method extends.","marker":"[39]"},{"why":"RNN-T loss function used in the Transducer decoder objective.","marker":"[40]"},{"why":"Multitask CTC loss used alongside RNN-T in the Conformer-T objective.","marker":"[41]"},{"why":"LibriSpeech corpus used as the clean speech training and evaluation data.","marker":"[42]"},{"why":"CHiME-4 noise recordings used to synthesize noisy training/test pairs and as real noisy test sets.","marker":"[43]"},{"why":"DEMAND dataset providing unseen noise types for out-of-domain generalization tests.","marker":"[44]"},{"why":"SpecAugment data augmentation used during acoustic feature training.","marker":"[45]"}],"fun_headline_variants":["Tri-stage training + tiny disentangler cut noisy ASR errors by 25.7%","1.7M-param noise splitter inside ASR reduces word errors by 25.7%","NoisyD-CT: disentangle noise with 1.7M params, slash WER by 25.7%","Clean-speech-like reps from noisy input: 1.7M-param module wins 25.7%"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that the frozen clean-trained Conformer encoder's representation space is one where mean-squared-error distance between noisy and clean representations tracks how much noise remains, so that minimizing L_CON suppresses noise without erasing linguistic information the decoder needs.","fun_headline_variants_meta":{"raw":{"variants":["Tri-stage training + tiny disentangler cut noisy ASR errors by 25.7%","1.7M-param noise splitter inside ASR reduces word errors by 25.7%","NoisyD-CT: disentangle noise with 1.7M params, slash WER by 25.7%","Clean-speech-like reps from noisy input: 1.7M-param module wins 25.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000582,"raw_usage":{"total_tokens":2625,"prompt_tokens":842,"completion_tokens":1783,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":1672}},"tokens_in":586,"tokens_out":1783,"duration_ms":15586,"temperature":1.0,"reasoning_tokens":1672,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:53:02.066867+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the frozen clean encoder that produces the regression target with a differently initialized Conformer encoder of the same size, keep every other detail fixed, and retrain only stages two and three; if the CHiME-4 WER gains disappear, the benefit depends on the teacher representation space rather than on the disentanglement structure itself.","supporting_citations":[{"cited_title":"Gulati, J","cited_arxiv_id":null,"evidence_quote":"Conformer encoder architecture whose frozen representations define the representation space the NoisyD module operates on."},{"cited_title":"Zhang, H","cited_arxiv_id":null,"evidence_quote":"Transducer formulation that, with Conformer, forms the baseline Conformer-Transducer the method extends."},{"cited_title":"Graves, Sequence transduction with recurrent neural networks, in: Proc","cited_arxiv_id":null,"evidence_quote":"RNN-T loss function used in the Transducer decoder objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Multitask CTC loss used alongside RNN-T in the Conformer-T objective."},{"cited_title":"Panayotov, G","cited_arxiv_id":null,"evidence_quote":"LibriSpeech corpus used as the clean speech training and evaluation data."},{"cited_title":"Vincent, S","cited_arxiv_id":null,"evidence_quote":"CHiME-4 noise recordings used to synthesize noisy training/test pairs and as real noisy test sets."},{"cited_title":"Thiemann, N","cited_arxiv_id":null,"evidence_quote":"DEMAND dataset providing unseen noise types for out-of-domain generalization tests."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SpecAugment data augmentation used during acoustic feature training."}],"review_version":1}