Conformer with bi-LSTM improves perceived naturalness of ultrasound-to-speech synthesis over a 2D-CNN baseline, while Conformer Base matches the baseline with faster training.
Conformer-based Ultrasound-to-Speech Conversion
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Deep neural networks have shown promising potential for ultrasound-to-speech conversion task towards Silent Speech Interfaces. In this work, we applied two Conformer-based DNN architectures (Base and one with bi-LSTM) for this task. Speaker-specific models were trained on the data of four speakers from the Ultrasuite-Tal80 dataset, while the generated mel spectrograms were synthesized to audio waveform using a HiFi-GAN vocoder. Compared to a standard 2D-CNN baseline, objective measurements (MSE and mel cepstral distortion) showed no statistically significant improvement for either model. However, a MUSHRA listening test revealed that Conformer with bi-LSTM provided better perceptual quality, while Conformer Base matched the performance of the baseline along with a 3x faster training time due to its simpler architecture. These findings suggest that Conformer-based models, especially the Conformer with bi-LSTM, offer a promising alternative to CNNs for ultrasound-to-speech conversion.
citation-role summary
background 1
citation-polarity summary
fields
cs.SD 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Conformer-based Ultrasound-to-Speech Conversion
Conformer with bi-LSTM improves perceived naturalness of ultrasound-to-speech synthesis over a 2D-CNN baseline, while Conformer Base matches the baseline with faster training.