Pith. sign in

REVIEW 3 major objections 8 minor 41 references

DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation

T0 review · 3 major / 8 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DAVE claims that speech separation in real-world audio-visual scenes is made robust by keeping video out of the reconstruction path and by post-enhancing only samples without ground-truth references.

desk verdict Solid challenge-system paper with a genuinely useful corpus and a sensible decoupled design, but the 'certified' non-degradation claim in Section 3.3 does not survive contact with the 98.13%-accurate router. read the letter →

arxiv 2608.09288 v1 pith:ARRT5BU2 submitted 2026-08-10 cs.SD cs.AI

Wei Zhou , Wanyi Ning , Yinshang Guo , Qianxiao Fang , Haitao Qian , Yingpeng Li This is my paper · ORCID
classification cs.SDcs.AI
keywords audio-visualspeechenhancementseparationtargetspeakerextractionreal-worldrobustnessvisualdegradationmulti-objectivetrainingdataaugmentationattribution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that real-world audio-visual speech separation can be made robust to unreliable visual input by decoupling the pipeline: an audio-only separation network reconstructs the speech streams, while visual information is reserved for a separate speaker-attribution vote. It further claims that a selective enhancement chain, made of scene routing, GAN-based denoising, and loudness normalization, can improve perception-oriented metrics without touching reference-based metrics, because enhancement runs only on samples that have no ground-truth reference. To support this, the authors build DAVE-Corpus, 219,411 realistic mixtures assembled from public meeting corpora with reverberation and noise augmentation, and train a scaled TIGER-M network under a progressive multi-objective loss covering separation, recognition, speaker identity, and perceptual quality. If the claims hold, a challenge-oriented system could gain perceived-quality improvements and visual-degradation robustness while leaving the scored reference metrics untouched by construction.

What carries the argument

The load-bearing mechanism is the decoupling architecture plus the certified selective enhancement chain. The chain begins with a lightweight acoustic scene classifier that splits samples into a no-reference partition and a reference partition; only the no-reference partition proceeds through layered GAN denoising and ITU-R BS.1770 loudness normalization, while reference-scored samples pass through unchanged. This makes non-degradation of SI-SDR, PESQ, and STOI a structural property of the pipeline rather than an empirical hope. The audio-only backbone keeps the separation itself independent of visual quality, and the weighted-fusion vote converts visual cues into a discrete attribution decision.

What would settle it

Take every development-set remix sample that the acoustic scene router labels as no-reference (the roughly 1.87% misrouting channel), run the full GAN-denoising and 6x loudness-normalization chain on them, and compare their SI-SDR, PESQ, and STOI with the untreated versions; a statistically reliable drop in any of these metrics on the misrouted set would refute the structural non-degradation guarantee.

Watch

Extended reading notes

Core claim

DAVE's central design choice is to reserve visual features for low-bandwidth decision-making instead of fusing them into the separation network. The audio branch, TIGER-M, is a scaled TIGER network with 2.56 million parameters trained from scratch on DAVE-Corpus with a permutation-invariant SI-SDR loss, a frozen-ASR cross-entropy loss, a speaker-fidelity cosine loss, and differentiable STOI, PESQ, and UTMOS losses. It outputs two anonymous streams, and a separate visual attribution module assigns identities through a weighted one-shot fusion of four votes: voiceprint similarity with weight 3.0, StableSyncNet audio-visual matching with 1.5, SyncNet synchrony with 1.0, and lip-keypoint motion with 0.7. A certified selective enhancement chain classifies each sample by acoustic scene statistics into remix or real-recording partitions, routing only no-reference samples through MossFormerGAN denoising and a 6x loudness normalization. On the official challenge DAVE reaches an SI-SDR of 10.23 dB, a CER of 0.171, and a speaker similarity of 0.726 on the real-world mixed track, and an SI-SDR of 8.93 dB with a CER of 0.220 on the visual degradation track.

Load-bearing premise

The structural guarantee that reference-based metrics cannot degrade depends on the acoustic scene router never sending a sample that will be scored against a reference into the enhancement chain, yet the router is only 98.13% accurate on the development set and the cases where it is wrong are not analyzed.

Editorial extensions

If this is right

  • Because the separation backbone never sees video, occlusion, blur, or missing face tracks cannot alter the reconstructed speech, which is what the visual-degradation track results reflect.
  • Training on DAVE-Corpus with the CER and speaker-fidelity losses lowers character error rate and raises speaker similarity relative to the SI-SDR-only baseline, while the perceptual losses raise PESQ, UTMOS, and DNSMOS-OVRL.
  • Restricting GAN denoising and loudness normalization to the no-reference partition means the UTMOS and DNSMOS gains on the development set come with the reference-based metrics unchanged by construction.
  • The one-shot weighted fusion of voiceprint, lip-sync, SyncNet, and keypoint votes can still assign speaker identity when individual visual experts are unreliable, because the voiceprint anchor carries the largest weight.
  • Scaling the backbone to 2.56 million parameters resolves the capacity saturation of the 0.82-million-parameter version, so additional training data continues to improve separation performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the certified guarantee is only as strong as the scene router: on out-of-distribution recordings its remix-error rate could rise, so the non-degradation promise should be re-audited whenever the acoustic domain changes.
  • A testable extension is to replace the hard routing decision with a confidence threshold, passing boundary-near samples through unchanged; this would trade a small perceptual gain for a formally stronger guarantee.
  • The same decoupling recipe could transfer to other multi-modal tasks where one modality is reliable for labeling but too fragile for low-level reconstruction: generate modality-agnostic hypotheses first, then use the fragile modality only to rank or select among them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. This paper presents DAVE, an audio-visual speech enhancement system for the Real-World Audio-Visual Speech Enhancement Challenge. The system decouples audio reconstruction from visual speaker attribution: an audio-only TIGER-M separator is trained on a newly constructed DAVE-Corpus with a multi-objective loss (permutation-invariant SI-SDR, CER, speaker similarity, and differentiable perceptual losses), while a visual branch assigns speaker identities by weighted fusion of voiceprint, lip-sync, and keypoint votes. A selective enhancement chain, described as certified, applies scene routing, GAN-based denoising, and loudness normalization only to samples classified as no-reference. The paper reports stepwise ablations on the development set and challenge leaderboard results for Track 1 (real-world mixed) and Track 2 (visual degradation), where DAVE is fourth on Track 1 and third on Track 2 by mean rank and clearly outperforms the official baseline.

Significance. If the claims were fully supported, the decoupled architecture would be a useful design pattern: the Track 2 results give real, if not top-ranked, evidence that an audio-only separation backbone remains usable when visual inputs are degraded, and the DAVE-Corpus construction addresses a genuine data-scarcity problem for realistic audio-visual training. The paper also provides a stepwise ablation that isolates the contribution of each training objective, which is valuable for practitioners. The central weakness is that the headline certified non-degradation guarantee is not established: the router is 98.13 percent accurate, leaving a misrouting channel that is never analyzed, and the CER loss shares its ASR model with the official CER evaluation. These two issues affect the main advertised contributions and require careful revision before the paper can be accepted.

major comments (3)
  1. [Section 3.3, Scene Routing] The claim that restricting enhancement to the routed no-reference partition yields a structural guarantee that reference-based metrics cannot degrade is not supported. The acoustic scene router is reported at 98.13 percent accuracy on the development set, so there is a 1.87 percent error channel. A remix sample with ground-truth references that is misrouted into the no-reference branch would receive MossFormerGAN denoising and 6x loudness normalization, and nothing in the paper demonstrates that those operations leave SI-SDR, PESQ, STOI, or CER unchanged. The manuscript does not report the number of misrouted remix samples, their per-sample metric deltas, or any worst-case bound. The guarantee should be restated as conditional on routing correctness, and the misrouting channel should be quantified or bounded before the word certified is used.
  2. [Section 3.1, Eq. (2)] The CER loss is computed with a frozen FunASR-Nano-2512 model, and Section 2 states that the same ASR model is used for both self-labeling the training data and official CER evaluation. This creates a circularity risk: optimizing the CER loss can improve the reported CER by fitting the idiosyncrasies of that particular ASR model rather than by genuinely improving intelligibility. The paper should either evaluate CER with an independent ASR model, report a cross-ASR comparison, or provide evidence that this overfitting effect is negligible. As written, the CER improvements in Tables 2 and 3 are not fully interpretable.
  3. [Section 3.3 and Table 2] The ablation reports that the five reference-based metrics are unchanged after certified enhancement, but the text says this is by construction rather than showing empirical verification. If any remix sample in the development set were misrouted, its reference metrics would change, so the exact equality in Table 2 is either evidence that routing was perfect on that subset or an artifact of rounding and aggregation. The paper should provide the routing confusion matrix on the development and test sets, the count of misrouted remix samples, and the per-sample deltas in reference metrics. In addition, the protocol assumption that the official evaluation confines reference metrics to the remix partition should be supported by an explicit citation to the challenge rules rather than asserted.
minor comments (8)
  1. [Section 2] Reference [15], Rabiner's HMM tutorial, is not a natural citation for frame-level energy screening; please cite an appropriate energy-based voice activity detection method.
  2. [Sections 2 and 4] The validation set is described as 150 held-out sessions in Data Construction but as 149 sessions in the Experiments section; please reconcile these numbers.
  3. [Section 3.1] The model name FunASR-Nano-2512 is not identifiable from the FunASR toolkit citation alone; please specify the exact checkpoint and version for reproducibility.
  4. [Section 3.2] The attribution weights (3.0, 1.5, 1.0, 0.7) and the speaker-similarity rejection threshold (0.5) are introduced without a sensitivity analysis or an ablation of the weighted fusion, which would strengthen the claim that this design is robust.
  5. [Section 3.3] The statement that loudness normalization is verifiably lossless with respect to the speaker-similarity metric is unclear, because scaling changes the waveform; please state explicitly which embedding invariance is claimed and under which peak-protection conditions.
  6. [Table 3] DAVE ranks fourth on Track 1 and third on Track 2 by mean rank, so the text should avoid any implication of overall superiority over the other leaderboard systems; the discussion should focus on the robustness comparison and the improvements over the official baseline.
  7. [Table 3] The leaderboard differences are reported without significance tests or confidence intervals; since the test set is fixed, a brief statement about the lack of repeated-evaluation uncertainty would help readers interpret the rank differences.
  8. [Section 3.3] Please clarify which evaluation metrics are computed on which partition (remix versus real-recording) and explain why CER, which also uses reference transcriptions, is not grouped with the reference-based metrics.

Circularity Check

2 steps flagged · score 6.0 of 10

The 'certified' non-degradation guarantee is definitional and depends on a fallible router; CER and speaker-similarity gains are obtained by optimizing the exact models used to score them.

  1. self definitional [Abstract and Section 3.3, 'Certified Selective Enhancement']
    "Only samples routed to the no-reference class enter the subsequent enhancement steps, while samples with ground-truth references are passed through unchanged. This yields a structural guarantee that the reference-based metrics cannot degrade."

    The guarantee is the routing policy itself, not a derived property. Reference metrics are defined only on the remix subset; by never modifying that subset, the metrics are unchanged by construction. The 'certified' claim therefore reduces to the assumption that the acoustic scene classifier is perfect, yet the paper reports 98.13% development accuracy, leaving a 1.87% misrouting channel through which remix samples receive GAN denoising and 6x loudness normalization. Non-degradation is not validated; it is assumed in the definition of selective enhancement.

  2. other [Section 2 (Data Construction) and Section 3.1 (Audio Separation), CER and speaker-fidelity losses]
    "we use a frozen Fun-ASR-Nano-2512 model to generate reference transcriptions for all training mixtures, providing the text labels required by the CER loss during training. ... The same ASR model is used for both self-labeling the training data and official CER evaluation ... where e(·) is the speaker embedding extracted by a frozen WeSpeaker ResNet34 model, which is the official speaker similarity evaluation model."

    The CER metric is not an independent intelligibility measure: the same FunASR model generates the training labels, supplies the differentiable CER loss, and serves as the official CER evaluator. The reported CER reduction is therefore a self-consistency score between the separator and one fixed ASR, optimized directly by the loss. Likewise, the speaker-fidelity loss minimizes cosine distance in the exact WeSpeaker embedding space used for official speaker-similarity scoring, so the SPK gains in Table 2 are the optimized objective itself rather than an external confirmation of speaker identity preservation.

full rationale

The paper contains substantial independent content: DAVE-Corpus construction, the audio-only backbone, the visual attribution fusion, and the external challenge leaderboard results (SI-SDR, PESQ, STOI) are not self-referential. However, the central 'certified' claim in the abstract and Section 3.3 is circular in the sense that non-degradation of reference-based metrics is guaranteed by definition (do not touch the remix partition) and is then asserted as a structural property despite the reported 98.13% router accuracy. The CER and speaker-similarity results are also partially forced because the same models are used as training losses and as evaluation metrics. These are not mere self-citations; they are instances where a headline result reduces to the construction of the objective or the partition. The misrouting channel is never quantified, so the 'cannot degrade' statement is an assumption, not a measured outcome. Overall circularity score: 6.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen augmentation ranges and fusion weights, on the assumed reliability of frozen pretrained models as evaluation proxies, on the official protocol confining reference metrics to the remix subset, and on the scene router being accurate enough to be treated as a guarantee. None of these is derived or externally validated in the paper.

free parameters (6)
  • Attribution voting weights = 3.0 (voiceprint), 1.5 (StableSyncNet), 1.0 (SyncNet), 0.7 (keypoints)
    Hand-chosen weights in Section 3.2; no tuning analysis or sensitivity study is reported, and they set the one-shot attribution decision.
  • Relative gain range for second source = [-3, 3] dB
    Data augmentation sampling range in Section 2; chosen without justification.
  • Noise SNR range = 12 to 30 dB
    MUSAN noise augmentation range in Section 2; hand-set.
  • Reverberation RT60 range = 0.2 to 0.4 s
    Room simulation range in Section 2; hand-set and narrow, may not cover real rooms.
  • Loudness normalization target = 6x original amplitude
    Section 3.3; chosen to improve DNSMOS-OVRL; no sweep reported.
  • Speaker-pair similarity rejection threshold = 0.5 cosine similarity
    Section 2; used to avoid pairing similar speakers; hand-set without reported sensitivity.
assumptions (4)
  • domain assumption The scene classifier is accurate enough that routing errors are negligible; the paper treats 98.13% accuracy as a structural guarantee.
    Section 3.3 asserts a structural guarantee of non-degradation after reporting only 98.13% classifier accuracy; the 1.87% error mode is not analyzed.
  • domain assumption Official evaluation confines reference-based metrics to the synthetic remix subset, so modifying only no-reference samples cannot affect them.
    Section 3.3 relies on this protocol assumption; if the protocol differs, the guarantee breaks.
  • domain assumption The frozen ASR and speaker embedding models used in the losses are faithful proxies for the official evaluation metrics.
    Section 3.1 uses FunASR-Nano-2512 for CER, WeSpeaker for speaker similarity, and official UTMOSv2 weights; correctness of the reported gains depends on these proxies.
  • domain assumption Visual evidence votes (voiceprint, lip sync, keypoints) can be normalized and weighted into a reliable one-shot attribution.
    Section 3.2 assumes weighted fusion of four evidence sources yields correct attribution with no confidence gating or fallback; no attribution accuracy is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation." pith.science (2026). https://pith.science/paper/ARRT5BU2

@misc{pith2026260809288,
  author       = {Pith},
  title        = {Pith review of: DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ARRT5BU2}},
  note         = {Machine review of arXiv:2608.09288}
}
read the original abstract

Audio-visual speech enhancement under real-world conditions remains challenging due to unreliable visual inputs and the lack of large-scale training data with realistic acoustic conditions. Existing approaches usually fuse visual features directly into the separation network, making them vulnerable to degraded visual signals. In this paper, we present DAVE, a decoupled audio-visual enhancement framework for real-world speech separation. Firstly, to address the data scarcity issue, we construct DAVE-Corpus, a large-scale training corpus with 219,411 mixtures generated from public meeting corpora through combinatorial acoustic augmentation. Then, we introduce a progressive multi-objective optimization strategy to jointly improve speech separation, intelligibility, speaker identity preservation, and perceptual quality. We further develop a certified selective enhancement chain that applies scene routing, GAN-based denoising, and loudness normalization only within the no-reference partition, guaranteeing non-degradation of reference-based metrics. Experimental results on the Real-World Audio-Visual Speech Enhancement Challenge demonstrate the robustness of DAVE under both real-world mixed scenarios and visual degradation conditions.

Figures

Figures reproduced from arXiv: 2608.09288 by the authors.

Figure 1
Figure 1. Overview of the DAVE framework. The audio￾only separation backbone reconstructs two anonymous speech streams, while a visual decision branch assigns speaker identi￾ties by weighted fusion of four evidence votes: voiceprint as the primary anchor, StableSyncNet, SyncNet, and lip-keypoint mo￾tion. The certified selective enhancement chain applies scene routing, GAN-based denoising, and loudness normalization be￾fore ou… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 35 canonical work pages

  1. [1]

    Introduction Audio-visual speech enhancement (A VSE) aims to recover the speech signal of a target speaker from noisy and interfering speech by exploiting complementary acoustic and visual in- formation. Compared with audio-only speech separation ap- proaches [1, 2, 3, 4], A VSE benefits from visual cues such as facial movements and speaker appearance, wh...

    work page

  2. [2]

    DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation

    Data Construction To train the audio separation backbone of DA VE under realistic acoustic conditions, we construct a large-scale audio training corpus, termed DA VE-Corpus, in three stages: synthetic mix- ture generation, official protocol remix, and EchoSet replay. Synthetic Mixture Generation.We collect speech seg- arXiv:2608.09288v1 [cs.SD] 10 Aug 202...

    work page Pith review arXiv 2026

  3. [3]

    As illustrated in Figure 1, the framework comprises three components

    Methodology DA VE decouples audio reconstruction from visual speaker attribution, reserving visual information for low-bandwidth decision-making tasks where it is most reliable. As illustrated in Figure 1, the framework comprises three components. First, an audio-only separation module is trained on the DA VE-Corpus with a progressive multi-objective stra...

    work page

  4. [4]

    Each GPU processes a batch of 2 three-second segments

    Experiments The TIGER-M model is trained from scratch on the DA VE- Corpus using 8×A800 GPUs with native PyTorch DDP. Each GPU processes a batch of 2 three-second segments. We use the Adam optimizer [35] with a peak learning rate of1×10 −3 and a linear warm-up of 1,000 steps, followed by ReduceL- ROnPlateau scheduling with patience of 3. The model is eval...

    work page arXiv

  5. [5]

    Conclusion We presented DA VE, a decoupled audio-visual enhancement framework for real-world speech separation that reserves visual information for reliable decision-making instead of direct fea- ture fusion. To address data scarcity, we built the DA VE-Corpus with 219,411 mixtures via combinatorial acoustic augmentation and room impulse response simulati...

    work page

  6. [6]

    Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,

    Y . Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 27, no. 8, pp. 1256– 1266, 2019

    work page 2019

  7. [7]

    Permutation invari- ant training of deep models for speaker-independent multi-talker speech separation,

    D. Yu, M. Kolbæk, Z.-H. Tan, and J. Jensen, “Permutation invari- ant training of deep models for speaker-independent multi-talker speech separation,” inProc. ICASSP, 2017

    2017

  8. [8]

    Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,

    J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,” inProc. ICASSP, 2016

    2016

Show all 41 references
  1. [9]

    Ps4: Proxy-supervised joint training for real target speaker extraction,

    W. Ning, W. Zhou, Y . Li, Y . Guo, H. Qian, and Y . Cheng, “Ps4: Proxy-supervised joint training for real target speaker extraction,”

  2. [10]

    Attention is all you need in speech separation,

    C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, and J. Zhong, “Attention is all you need in speech separation,” inProc. ICASSP, 2021

    2021

  3. [11]

    Looking to listen at the cock- tail party: A speaker-independent audio-visual model for speech separation,

    A. Ephrat, I. Mosseri, O. Lang, T. Dekel, K. Wilson, A. Hassidim, W. T. Freeman, and M. Rubinstein, “Looking to listen at the cock- tail party: A speaker-independent audio-visual model for speech separation,”ACM Trans. Graph., vol. 37, no. 4, 2018

    2018

  4. [12]

    The conversation: Deep audio-visual speech enhancement,

    T. Afouras, J. S. Chung, and A. Zisserman, “The conversation: Deep audio-visual speech enhancement,” inProc. Interspeech, 2018

    2018

  5. [13]

    The fifth CHiME speech separation and recogni- tion challenge: Dataset, task and baselines,

    J. Barkeret al., “The fifth CHiME speech separation and recogni- tion challenge: Dataset, task and baselines,” inProc. Interspeech, 2018

    2018

  6. [14]

    Time domain audio visual speech separation,

    J. Wuet al., “Time domain audio visual speech separation,” in Proc. ASRU, 2019

    2019

  7. [15]

    Dual-path RNN: Efficient long sequence modeling for time-domain single-channel speech sepa- ration,

    Y . Luo, Z. Chen, and T. Yoshioka, “Dual-path RNN: Efficient long sequence modeling for time-domain single-channel speech sepa- ration,” inProc. ICASSP, 2020

    2020

  8. [16]

    Tiger: Time-frequency interleaved gain extraction and reconstruction for efficient speech separation,

    L. Xu, C. Li, and R. Hu, “Tiger: Time-frequency interleaved gain extraction and reconstruction for efficient speech separation,” in Proc. ICASSP, 2025

    2025

  9. [17]

    The AliMeeting corpus: A multi-modal meeting corpus,

    Z. Duet al., “The AliMeeting corpus: A multi-modal meeting corpus,” inProc. ICASSP, 2022

    2022

  10. [18]

    MISP: A multi-modal interactive speech process- ing system,

    G. Wanget al., “MISP: A multi-modal interactive speech process- ing system,” inProc. Interspeech, 2021

    2021

  11. [19]

    AISHELL-4: An open source dataset for speech separation, diarization and recognition,

    Y . Fuet al., “AISHELL-4: An open source dataset for speech separation, diarization and recognition,” inProc. ICASSP, 2021

    2021

  12. [20]

    Pyroomacoustics: A Python package for audio room simulation and array processing algorithms,

    R. Scheibler, E. Bezzam, and I. Dokmani ´c, “Pyroomacoustics: A Python package for audio room simulation and array processing algorithms,” inProc. ICASSP, 2018

    2018

  13. [21]

    A tutorial on hidden markov models and selected applications in speech recognition,

    L. R. Rabiner, “A tutorial on hidden markov models and selected applications in speech recognition,”Proceedings of the IEEE, 1989

    1989

  14. [22]

    WeSpeaker: A research and production oriented speaker embedding learning toolkit,

    H. Wang, C. Liang, S. Wanget al., “WeSpeaker: A research and production oriented speaker embedding learning toolkit,” inProc. ICASSP, 2023

    2023

  15. [23]

    FunASR: A fundamental end-to-end speech recog- nition toolkit,

    Z. Gaoet al., “FunASR: A fundamental end-to-end speech recog- nition toolkit,” inProc. Interspeech, 2023

    2023

  16. [24]

    Room impulse response generator,

    E. A. Habets, “Room impulse response generator,”Technische Universiteit Eindhoven, Tech. Rep, vol. 2, no. 2.4, p. 1, 2006

    2006

  17. [25]

    MUSAN: A music, speech, and noise corpus,

    D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,” inarXiv:1510.08484, 2015

    2015 arXiv

  18. [26]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Des- jardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska- Barwinskaet al., “Overcoming catastrophic forgetting in neural networks,”Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  19. [27]

    SDR— half-baked or well done?

    J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR— half-baked or well done?” inProc. ICASSP, 2019

    2019

  20. [28]

    Out of time: automated lip sync in the wild,

    J. S. Chung and A. Zisserman, “Out of time: automated lip sync in the wild,” inACCV Workshops, 2016

    2016

  21. [29]

    An al- gorithm for intelligibility prediction of time-frequency weighted noisy speech,

    C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An al- gorithm for intelligibility prediction of time-frequency weighted noisy speech,”IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 7, pp. 2125–2136, 2011

    2011

  22. [30]

    Per- ceptual evaluation of speech quality (PESQ)—a new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Per- ceptual evaluation of speech quality (PESQ)—a new method for speech quality assessment of telephone networks and codecs,” in Proc. ICASSP, 2001

    2001

  23. [31]

    UTMOS: UTokyo-SaruLab system for V oiceMOS challenge 2022,

    T. Saekiet al., “UTMOS: UTokyo-SaruLab system for V oiceMOS challenge 2022,” inProc. Interspeech, 2022

    2022

  24. [32]

    CN-Celeb: A challenging Chinese speaker recognition dataset,

    Y . Fan, J. W. Kang, L. T. Li, K. C. Li, H. L. Chen, S. T. Cheng, P. Y . Zhang, Z. Y . Zhou, Y . Q. Cai, and D. Wang, “CN-Celeb: A challenging Chinese speaker recognition dataset,” inProc. ICASSP, 2020, pp. 7604–7608

    2020

  25. [33]

    LatentSync: Taming audio-conditioned latent diffusion models for lip sync with SyncNet supervision,

    C. Li, C. Zhang, W. Xu, J. Xie, W. Feng, B. Peng, and W. Xing, “LatentSync: Taming audio-conditioned latent diffusion models for lip sync with SyncNet supervision,”arXiv preprint arXiv:2412.09262, 2024

    2024 arXiv

  26. [34]

    DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise sup- pressors,

    C. K. Reddy, V . Gopal, and R. Cutler, “DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise sup- pressors,” inProc. ICASSP, 2021

    2021

  27. [35]

    Perfect match: Improved cross-modal embeddings for audio-visual synchronisa- tion,

    S.-W. Chung, J. S. Chung, and H.-G. Kang, “Perfect match: Improved cross-modal embeddings for audio-visual synchronisa- tion,” inProc. ICASSP, 2019, pp. 3965–3969

    2019

  28. [36]

    How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks),

    A. Bulat and G. Tzimiropoulos, “How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks),” inProc. ICCV, 2017, pp. 1021–1030

    2017

  29. [37]

    SEGAN: Speech en- hancement generative adversarial network,

    S. Pascual, A. Bonafonte, and J. Serr `a, “SEGAN: Speech en- hancement generative adversarial network,” inProc. Interspeech, 2017

    2017

  30. [38]

    MossFormer2: Combining transformer and RNN-free recurrent network for enhanced time-domain monaural speech separation,

    S. Zhaoet al., “MossFormer2: Combining transformer and RNN-free recurrent network for enhanced time-domain monaural speech separation,” inProc. ICASSP, 2024

    2024

  31. [39]

    Recommendation ITU- R BS.1770-4: Algorithms to measure audio programme loudness and true-peak audio level,

    International Telecommunication Union, “Recommendation ITU- R BS.1770-4: Algorithms to measure audio programme loudness and true-peak audio level,” ITU, 2015

    2015

  32. [41]

    Adam: A method for stochastic opti- mization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic opti- mization,” inProc. International Conference on Learning Repre- sentations (ICLR), 2015

    2015

  33. [2026]

    Available: https://arxiv.org/abs/2607.08111

    [Online]. Available: https://arxiv.org/abs/2607.08111

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.