{"id":"a6116dc1-d005-4aca-9684-e443feea54a7","arxiv_id":"2505.05077","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ReverbMiipher, an extension of Miipher-2 with a dedicated reverb encoder, preserves and controls reverberation during generative speech restoration.","lead":"A speech restoration model called ReverbMiipher cleans noisy audio while keeping the original room echo, and can even change that echo to a different room sound. The work adds a learned reverb knob to the Miipher-2 restoration pipeline, which could make AR/VR and video audio feel more realistic.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Disentanglement of the reverb feature is asserted, not demonstrated: Eq. (1)'s zero-vector switch does not prevent c from leaking speaker/content, so the controllability results are confounded.","rationale":"The reader's weakest assumption identifies exactly the load-bearing gap: the stochastic zero-vector replacement is supposed to make c encode only reverberation, but the paper provides no quantitative disentanglement evidence. My reading of Eq. (1) supports this concern further: the switching term only defines the model's behavior when c is the zero vector; it does not regularize the content of nonzero c. Because the vocoder has access to both h_s and c, there is no optimization pressure to remove speaker, content, or noise information from c unless an explicit penalty or adversary is added, and none is described. The subjective ranking (mean rank 1.49, p<1e-8) and the MCD/SPK-sim improvements in Table 2 are real evidence that ReverbMiipher preserves the reverberant target better than the two-stage baseline, but they are computed against s*r and therefore cannot distinguish 'c contains only reverb' from 'c contains enough speaker/content to help reconstruct s*r.' The interpolation and sampling demonstrations are compelling qualitatively, and the online audio samples are a useful resource, but they are not a quantitative test of disentanglement. This concern does not invalidate the paper's main restoration result; it specifically threatens the stronger controllability claim. Since the reader already marked the verdict CONDITIONAL, my read does not change that verdict: the paper should be accepted only if the disentanglement of c is verified or the controllability claims are softened accordingly.","tokens_in":9683,"tokens_out":4842,"duration_ms":49674,"concrete_test":"Use the released demo checkpoint (or if unavailable, retrain the model) and run a c-swap experiment. Fix an anechoic source utterance s (e.g., from EARS). Extract c_A from a reverberant utterance by speaker A and c_B from a reverberant utterance by speaker B, matching RT60 and DRR within a small tolerance. Synthesize y_A = V(h_s, c_A) and y_B = V(h_s, c_B). Measure SPK-sim(y_A, s) vs SPK-sim(y_B, s) and a content metric (e.g., WER) on both outputs. If the two outputs differ significantly in speaker or content while their estimated reverberation characteristics match, then c leaks non-reverb information and the controllability claim must be restricted. Repeat with c extracted from clean versus noisy versions of the same reverberant utterance to test noise leakage. This directly probes the disentanglement premise of Eq. (1).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ReverbMiipher preserves and controls only reverberation rests on the assumption that the 512-dim reverb feature c is disentangled from speaker identity, content, and noise. Section 3.2's stochastic zero-vector replacement, Eq. (1), does not actually enforce this. With probability q=0.1 the model is trained to output clean s when c=0, which teaches the vocoder how to interpret the zero vector; it imposes no constraint on the information content of nonzero c. Since the vocoder also receives h_s (which, by the paper's own admission in §3.2, is only 'expected' to contain speech information and not guaranteed to be free of reverberation), there is no gradient pressure preventing c from carrying speaker, content, or residual noise. The sentence 'Consequently, the ReverbEncoder weights are expected to be updated to selectively preserve information pertaining only to reverberation' is an assertion, not a demonstrated property. The evaluations in §4 compare outputs to s*r, so a c that leaks content or speaker could still achieve low MCD and high SPK-sim; those metrics do not isolate whether c encodes only reverberation. Section 5's interpolation and PCA-sampling demonstrations are qualitative spectrograms (Figs. 4-5) and do not measure whether speaker or content drift with α. If c leaks, the 'novel reverberation' demonstrations may be partially changing the speaker or wording while appearing to control only reverberation, which would undermine the headline controllability claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReverbMiipher, an extension of the Miipher-2 speech restoration model that adds a ReverbEncoder producing a 512-dimensional utterance-level feature c from the noisy log-mel input. The vocoder is conditioned on c, and training uses stochastic zero-vector replacement so that setting c to the zero vector yields clean speech while using the extracted c yields speech with reverberation. Experiments compare ReverbMiipher against Miipher-2 and a two-stage baseline (Miipher-2 followed by convolution with an oracle-parameter simulated RIR) using pairwise subjective ranking and objective MCD/GPE/SPK-sim against the reverberant reference. The paper also presents PCA visualization, linear interpolation of c, and sampling from the PCA plane as reverberation control applications.","tokens_in":9989,"tokens_out":5418,"duration_ms":52162,"significance":"If the claimed disentanglement holds, ReverbMiipher is a meaningful step toward speech restoration that preserves and controls the acoustic environment rather than removing it. The strongest evidence is the subjective ranking (mean rank 1.49 versus 2.01 and 2.50, p < 1e-8) and the improvement in MCD and SPK-sim over an upper-bound two-stage baseline that is given oracle RT60 and DRR values. The proposed use of a learned reverb feature with interpolation and PCA sampling is a practical and potentially generalizable interface. However, the paper's central controllability claim rests on an untested disentanglement assumption, and the artifact-removal claim lacks direct objective metrics.","major_comments":[{"comment":"The stochastic zero-vector replacement does not enforce the claimed disentanglement. In the branch U(0,1) <= q, the loss L_V(s, V(h_s, 0)) updates only the vocoder, not the ReverbEncoder, since c is fixed to 0. In the other branch, the ReverbEncoder is trained solely to minimize L_V(s*r, V(h_s, c)); any information in c that helps reconstruct the reverberant target, including speaker identity, content, or residual noise, is useful for this loss. The statement that artifacts in c 'will be treated as noise by the network' (§3.2) is an assertion without gradient-level justification. Since h_s is an imperfect estimate, c has a strong incentive to carry content and speaker information as a shortcut. Consequently, the interpolation and sampling demonstrations in §5 may change speaker or content rather than reverberation only. The authors should provide a quantitative disentanglement test, e.g., swap or interpolate c between utterances with different speakers or content and measure speaker similarity and word error rate of the output, or add an explicit information-bottleneck or adversarial objective.","section":"§3.2, Eq. (1)"},{"comment":"The abstract and §1 claim that ReverbMiipher 'removes other artifacts' such as noise and codec distortions, but Table 2 reports only MCD, GPE, and SPK-sim computed against the reverberant reference s*r. These metrics do not directly measure artifact removal (e.g., noise audibility, codec distortion) and are also consistent with c leaking content or speaker information, since such leakage would help match s*r. I recommend adding standard restoration metrics such as PESQ, STOI, or DNSMOS against the clean reference (or a no-reference metric), and possibly a subjective quality MOS, so that the artifact-removal claim is supported independently of the reverberation-preservation claim.","section":"§4.4, Table 2"},{"comment":"The controllability experiments are qualitative only. Fig. 4b and Fig. 5 show spectrograms and state that harmonic structure or original speech structure is preserved, but no objective measurement of content or speaker consistency is reported. Given the disentanglement concern in §3.2, these demonstrations do not establish that varying alpha or sampling c controls only reverberation. I suggest measuring, across the interpolation path alpha in [0,1] and across sampled points, word error rate (or phoneme error rate) and speaker embedding cosine similarity against the source utterance, as well as an objective reverb-parameter estimate (e.g., RT60 and DRR) of the output.","section":"§5.2–§5.3, Figs. 4–5"}],"minor_comments":[{"comment":"There are minor typos: 'V orbis' should be 'Vorbis' in §4.2, and 'better than to the baseline' should be 'better than the baseline' in §6.","section":"§4.2, §6"},{"comment":"The objective results are reported without error bars or confidence intervals; with 1070 evaluation utterances, pairwise significance tests would strengthen the objective comparison.","section":"Table 2"},{"comment":"Clarify the treatment of anechoic samples, since RT60 is nominally 0 and DRR is undefined for an anechoic condition; the text says anechoic samples cluster at upper right, which is confusing given the axis ranges.","section":"Fig. 3"},{"comment":"Typo: 'uttrances' should be 'utterances', and 'Let c1 and c2 be a reverb features' should be 'Let c1 and c2 be reverb features'.","section":"§5.2"},{"comment":"The switching probability q is not ablated; a brief sensitivity study or a justification for q=0.1 would help, since q controls the tradeoff between preservation and controllability.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"I do not see any indication of misconduct or missing attribution. The main risk is that the headline 'disentangled reverb feature' claim is not supported by the current metrics; if the authors add a quantitative disentanglement check and standard artifact-removal metrics, the paper would be suitable for publication. The paper is within the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here is my read on ReverbMiipher (arXiv:2505.05077). The core idea is simple: add a ReverbEncoder to Miipher-2, condition the vocoder with a 512-dim reverb vector, and during training randomly replace that vector with zeros so the model occasionally has to output clean speech. The subjective ranking (2,878 ratings) and the MCD/SPK-sim numbers in Table 2 do show that ReverbMiipher preserves reverberation better than Miipher-RIR, a baseline that is actually given an oracle RIR. That is a credible result, and the architecture is a natural extension rather than a wild leap. Credit where due: the PCA visualization in Fig. 3 shows the feature varies smoothly with RT60 and DRR, which supports the idea that the feature is at least reverb-related, and the paper is honest about the baseline being an upper bound on what real RIR estimation could achieve. My main concern is the disentanglement claim. Section 3.2 says the stochastic zero-vector replacement 'ensures' the feature encodes only reverberation, but Eq. (1) only trains the vocoder to interpret a zero vector as 'no reverb'. It imposes no constraint on what nonzero c can carry. The paper's own wording in §3.2 is hedged ('expected to be updated'), which is accurate, because there is no gradient pressure preventing c from leaking speaker identity, content, or residual noise. The objective metrics compare against s*r, so leakage that actually helps reconstruct the reverberant target would not be penalized. The controllability demonstrations in Section 5 are qualitative spectrograms; nothing there shows that interpolation changes only the reverb and not the speaker or wording. If c leaks, the 'novel reverberation' demos could be partially changing other attributes while appearing to control only reverb. This is not fatal to the preservation claim, but it does undermine the headline controllability claim as stated. A second, smaller gap: the paper says ReverbMiipher 'removes other artifacts' but never reports standard enhancement metrics like PESQ or STOI. The evidence is indirect (subjective resemblance to s*r). Given the architecture, the artifact removal is plausible, but it is not measured. Reproducibility is also partial: USM, WaveFit, the internal noise dataset, and the trained model are not released, though demo audio is. For a speech-restoration audience, this is worth engaging with. The reverb-preservation result is a genuine addition, and the training trick is worth discussing even if the disentanglement needs sharper verification. I would send it to peer review and ask the authors to do two things: include feature-leakage checks (e.g., show that speaker/content metrics are unchanged across c interpolation or swapping), and add at least one conventional speech quality metric like PESQ to back the artifact-removal claim. If they can supply a disentanglement analysis, this could become a solid conditional accept. If not, it remains a useful empirical contribution with an over-claimed controllability narrative. Recommended for a serious referee.","headline":"ReverbMiipher is a solid incremental extension of Miipher-2 that beats a two-stage RIR baseline at preserving reverberation, but the claimed disentanglement of the reverb feature is asserted, not demonstrated.","tokens_in":10508,"tokens_out":2029,"would_cite":true,"duration_ms":22826,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a generative speech-restoration model can remove noise and codec artifacts while preserving the original reverberation, and that it does so by routing a dedicated reverb feature through the vocoder, outperforming the…","keywords":["speech restoration","reverberation preservation","reverberation control","parametric resynthesis","reverb feature disentanglement","neural vocoder","latent space interpolation","room impulse response"],"falsifier":"Take a fixed clean anechoic utterance s, extract reverb features c_i from a set of noisy utterances with different speakers and content, and synthesize V(ĥ_s, c_i). If the reverb feature is truly disentangled, the output should have the same ASR transcript and near-identical speaker embedding as s for every c_i, with only the room acoustics changing; a systematic drop in speaker similarity or rise in word error rate as c_i varies would falsify the disentanglement claim.","tokens_in":9500,"feed_emoji":"🎙️","tokens_out":5260,"duration_ms":50060,"temperature":0.7,"pith_summary":"Speech-restoration systems typically erase the reverberation that tells listeners about the room the speaker was in. ReverbMiipher extends the Miipher-2 parametric-resynthesis architecture with a ReverbEncoder that reads a log-mel spectrogram of the noisy input and produces a 512-dimensional reverb feature; this feature conditions the WaveFit vocoder together with the restored speech features, so the model learns to map the noisy signal to the reverberant clean signal s*r rather than to anechoic speech. A stochastic zero-vector replacement during training teaches the model that a zero vector means 'no reverb', which is what pushes the encoder to encode only reverberation. The paper reports that this one-stage model preserves reverberation better, by subjective ranking and by objective metrics, than restoring clean speech and then convolving a simulated room impulse response, and that interpolation or PCA-plane sampling of the reverb feature yields continuous control over the acoustic environment.","feed_headline":"Reverb survives cleanup in new speech-restoration model","feed_subtitle":"A dedicated reverb feature lets the model clean audio yet keep the original room acoustics.","key_machinery":"The central object is the 512-dimensional reverb feature c = R(x) produced by the ReverbEncoder, a stack of four Conformer layers with kernel size five followed by time-axis mean pooling, operating on the same 128-band log-mel input as the feature cleaner. This vector conditions the WaveFit vocoder as V(ĥ_s, c), and the training loss of Eq. (1) switches stochastically with probability q = 0.1 between the reverberant target s*r (when c is the true encoder output) and the anechoic target s (when c is a zero vector). The switching is what forces the encoder to specialize: the vocoder must learn to use c to reproduce the room, while the zero-vector case tells it that no reverb is also a valid condition, so any non-reverb information in c would have to be ignored or would corrupt the anechoic output. That mechanism is what separates the reverb representation from the speech content and makes downstream interpolation and sampling meaningful.","core_discovery":"ReverbMiipher's central claim is that reverberation can be treated as an explicit conditioning variable inside a generative speech-restoration model rather than something to remove or add afterwards. The ReverbEncoder's feature vector c is the carrier of the room acoustics: when c is present, the vocoder is trained to output s*r, the clean speech convolved with the original room impulse response, and when c is the zero vector, it must output the anechoic source s. Because the model receives only the noisy input, the encoder must pull the reverb information out of the degraded mixture, and the random switching with probability q = 0.1 is the training signal that prevents c from smuggling in speaker identity, content, or noise. The paper then shows that c varies smoothly with RT60 and DRR in a PCA projection, that linear interpolation between two c vectors produces intermediate reverberation, and that sampling from the PCA plane gives new but structurally intact reverberation effects. The claim is that this one-stage, feature-conditioned design outperforms the two-stage baseline of Miipher-2 restoration followed by convolution with a simulated RIR chosen with known RT60 and DRR.","pith_inferences":["If the disentanglement holds, the same conditioning trick could be extended to other acoustic attributes, such as environmental sounds, effectively turning speech restoration into a controllable acoustic-scene remixer.","A testable extension is cross-speaker or cross-language reverb transfer: swapping the reverb feature between utterances with different speakers or content should leave the transcript and speaker identity untouched, which can be verified with ASR and speaker-embedding similarity.","The smooth PCA-plane structure suggests the reverb feature manifold may be low-dimensional; an explicit dimensionality analysis or a learned controllable latent could give even finer control than linear interpolation.","The subjective evaluation uses pairwise ranking focused on reverberation similarity, so a future test that also measures overall perceived quality would clarify whether reverb preservation comes at any cost to other perceptual dimensions."],"forward_implications":["Reverb preservation becomes a built-in behavior of speech restoration rather than a post-hoc step, so the acoustic environment of the original recording is retained in applications such as augmented reality, virtual reality, and video generation.","The reverb feature can be transferred between utterances, interpolated, or sampled from a PCA plane, enabling the creation of datasets with controllable spatial characteristics.","Because the model is trained end-to-end rather than relying on explicit RIR estimation, it avoids the error propagation and fidelity losses of the two-stage clean-then-convolve pipeline.","The approach beats the simulated-RIR baseline even when that baseline is given perfect knowledge of RT60 and DRR, indicating that the advantage comes from the architecture itself rather than from having better room information."],"supporting_citations":[{"why":"Supplies the parametric resynthesis framework that ReverbMiipher extends.","marker":"[13]"},{"why":"Miipher-2 is the base speech-restoration model whose feature cleaner and vocoder architecture ReverbMiipher inherits and modifies.","marker":"[26]"},{"why":"Provides the pre-trained Universal Speech Model used as the clean-feature target for the feature cleaner.","marker":"[42]"},{"why":"WaveFit is the vocoder that is conditioned on the reverb feature to produce the output waveform.","marker":"[45]"},{"why":"The image method generates the simulated RIRs used for training data augmentation and for the Miipher-RIR baseline.","marker":"[29]"},{"why":"Pyroomacoustics implements the RIR simulation used in training and in the baseline RIR selection.","marker":"[46]"},{"why":"FLEURS-R is the training corpus of clean multilingual speech from which noisy training samples are created.","marker":"[48]"},{"why":"EARS provides the anechoic speech used to build the evaluation set.","marker":"[49]"},{"why":"WHAM! supplies the noise sources mixed into the evaluation signals.","marker":"[50]"},{"why":"The MIT survey provides the real-recorded room impulse responses used to create the reverberant evaluation conditions.","marker":"[51]"}],"fun_headline_variants":["Speech cleanup that keeps the room sound intact","New model restores speech without stripping reverb","Reverb becomes a dial in generative speech restoration","One-stage denoising with full reverb control","Restore speech, keep the acoustics: ReverbMiipher"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole control scheme rests on the assumption that training with a 10% zero-vector switching probability makes the reverb feature encode only reverberation; if that vector also leaks speaker identity or content, the demonstrated interpolation and sampling are changing the speech itself rather than just the room.","fun_headline_variants_meta":{"raw":{"variants":["Speech cleanup that keeps the room sound intact","New model restores speech without stripping reverb","Reverb becomes a dial in generative speech restoration","One-stage denoising with full reverb control","Restore speech, keep the acoustics: ReverbMiipher"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1239,"prompt_tokens":952,"completion_tokens":287,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":211}},"tokens_in":568,"tokens_out":287,"duration_ms":3145,"temperature":1.0,"reasoning_tokens":211,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:13:16.720011+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed clean anechoic utterance s, extract reverb features c_i from a set of noisy utterances with different speakers and content, and synthesize V(ĥ_s, c_i). If the reverb feature is truly disentangled, the output should have the same ASR transcript and near-identical speaker embedding as s for every c_i, with only the room acoustics changing; a systematic drop in speaker similarity or rise in word error rate as c_i varies would falsify the disentanglement claim.","supporting_citations":[{"cited_title":"Parametric resynthesis with neural vocoders,","cited_arxiv_id":null,"evidence_quote":"Supplies the parametric resynthesis framework that ReverbMiipher extends."},{"cited_title":"Google USM: Scaling automatic speech recognition beyond 100 languages,","cited_arxiv_id":null,"evidence_quote":"Provides the pre-trained Universal Speech Model used as the clean-feature target for the feature cleaner."},{"cited_title":"WaveFit: An iterative and non- autoregressive neural vocoder based on fixed-point iteration,","cited_arxiv_id":null,"evidence_quote":"WaveFit is the vocoder that is conditioned on the reverb feature to produce the output waveform."},{"cited_title":"Image method for efficiently simulating small-room acoustics,","cited_arxiv_id":null,"evidence_quote":"The image method generates the simulated RIRs used for training data augmentation and for the Miipher-RIR baseline."},{"cited_title":"Pyroomacoustics: A python package for audio room simulation and array processing algorithms,","cited_arxiv_id":null,"evidence_quote":"Pyroomacoustics implements the RIR simulation used in training and in the baseline RIR selection."},{"cited_title":"FLEURS-R: A restored multilingual speech corpus for generation tasks,","cited_arxiv_id":null,"evidence_quote":"FLEURS-R is the training corpus of clean multilingual speech from which noisy training samples are created."},{"cited_title":"EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and dereverberation,","cited_arxiv_id":null,"evidence_quote":"EARS provides the anechoic speech used to build the evaluation set."},{"cited_title":"WHAM!: Extending speech separation to noisy environments,","cited_arxiv_id":null,"evidence_quote":"WHAM! supplies the noise sources mixed into the evaluation signals."},{"cited_title":"Statistics of natural reverberation enable perceptual separation of sound and space,","cited_arxiv_id":null,"evidence_quote":"The MIT survey provides the real-recorded room impulse responses used to create the reverberant evaluation conditions."}],"review_version":1}