Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model

Pierre-Andr\'e Vuissoz; Sofiane Azzouz; Yves Laprie

REVIEW 4 major objections 6 minor 40 references

Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model

T0 review · 4 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Clean speech inverts to vocal-tract shapes almost as well as MRI audio

desk verdict A careful single-speaker feasibility study: C2C inversion comes close to M2M, but the absolute RMSE is a cross-session consistency measure, and the paper should disclose alignment-set details. read the letter →

arxiv 2603.11845 v2 pith:DRGYJPX4 submitted 2026-03-12 eess.AS

Sofiane Azzouz , Pierre-André Vuissoz , Yves Laprie This is my paper · ORCID

classification eess.AS

keywords acoustic-to-articulatoryinversionreal-timeMRIvocaltractcontourcleanspeechHuBERTembeddingsphoneticalignmentproductionBi-LSTM

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This study asks whether articulatory inversion—reconstructing vocal-tract shape from audio—can work on clean recordings instead of denoised MRI audio. The authors compare three conditions: MRI audio for both training and testing (M2M), MRI-trained on clean speech (M2C), and clean speech for both (C2C). Their central result is that the clean-only model reaches a mean contour error of 1.56 mm, nearly matching the 1.51 mm MRI baseline. This matters because it suggests speech recorded outside the scanner is usable for real-world articulatory inversion, bypassing scanner noise and denoising. The close numbers depend on a phoneme-level time-stretching alignment that assumes MRI and clean utterances share the same articulatory targets.

What carries the argument

The key mechanism is a hierarchical phonetic alignment: sentences, words, and phonemes in the clean corpus are matched to the MRI corpus, and each MRI frame is mapped to a clean-speech frame via intra-phoneme relative position and time-stretching (Eqs. 2–5). This produces paired audio-contour training targets without requiring simultaneous MRI and clean recordings. The inversion model itself is a five-layer network—two dense layers, two Bi-LSTM layers, and a dense output—that maps 768-dimensional HuBERT-Base speech embeddings to 8 articulator contours of 50 points each; contours come from automatic tracking of rt-MRI frames.

What would settle it

Take the same speaker, record the same sentences in the MRI scanner and in a quiet room, and independently measure articulator positions during the quiet-room speech (for example with electromagnetic articulography). If the measured contours differ from the MRI-mapped targets by much more than 1.56 mm, the close RMSE in the paper reflects alignment consistency, not true clean-speech inversion accuracy.

Watch

Extended reading notes

Core claim

The paper claims that a model trained and evaluated on clean speech (C2C) reconstructs eight vocal-tract contours with an average RMSE of 1.56 mm, compared to 1.51 mm for the denoised MRI baseline (M2M), and significantly better than the 1.64 mm obtained when an MRI-trained model is evaluated on clean speech (M2C). It interprets this as evidence that clean speech supports articulatory inversion effectively and that the proposed phoneme-alignment procedure is what makes clean speech usable.

Load-bearing premise

The load-bearing assumption is that after phoneme-level time-stretching, the same sentence uttered in the MRI scanner and in a quiet room corresponds to the same vocal-tract target shapes—ignoring supine posture and Lombard-effect changes that the paper explicitly sets aside.

Editorial extensions

If this is right

If 1.56 mm is a stable error, clean-speech inversion is accurate enough for visual articulatory feedback applications, since the MRI pixel size is 1.62 mm and the errors are at sub-pixel level.
Phonetic-segmentation-based alignment outperforms raw DTW (1.56 vs 1.68 mm mean RMSE for C2C), so future cross-corpus inversion should reuse phoneme-boundary alignment rather than acoustic warping.
The M2C gap (1.51→1.64 mm) shows that even high-quality denoising leaves a measurable acoustic mismatch; training or adapting on clean speech is the more direct remedy.
Per-articulator results indicate tongue reconstruction is the hardest (2.46 mm in C2C) and lip/pharyngeal-wall the easiest, guiding which articulators to trust in applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves open whether the 1.56 mm would survive cross-corpora generalization; a natural test is to record the same speaker in the scanner and in quiet on non-identical sentences and see whether phoneme-level alignment still yields matching contours.
Because the ground-truth contours come from an automatic tracker on MRI, the reported error includes that tracker's own noise; an independent articulatory measurement (e.g., electromagnetic articulography) for clean speech would give the true inversion accuracy.
If clean-speech training can match MRI-trained performance, one could build inversion systems without any scanner data by using an MRI-derived articulatory atlas as the output target—provided the alignment assumption holds for new speakers.
The significant C2C vs M2C improvement suggests domain mismatch is mostly about acoustic style, not articulatory content; this predicts that a small amount of clean-speech fine-tuning of an MRI-trained model would close the gap — something the paper does not test.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is a solid, modest feasibility study, not a breakthrough. The new empirical result is that a model trained and tested on quiet-room speech (C2C) reaches 1.56 mm RMSE, only 0.05 mm above the MRI-trained baseline (M2M, 1.51 mm), and notably better than the mismatched M2C condition (1.64 mm). The phoneme-based alignment procedure and the DTW comparison are the real additions; the architecture and contour extraction are from the authors' prior work.

What the paper does well: single speaker, identical sentences across conditions, manually corrected phonetic segmentations, held-out split, paired t-tests. It also openly says it does not model Lombard/supine effects, which is a key limitation.

Now the soft spots, in order of importance. First, the ground truth for all conditions is the MRI session's contours. When clean speech is aligned to those contours, the reported RMSE is a measure of consistency between the model's prediction and the MRI-derived target, not a direct measure of inversion accuracy for clean production. If the speaker's articulation differed between the scanner and the quiet room, the true error could be larger than 1.56 mm. The paper acknowledges the unmodeled factors but does not quantify their impact. This limits the absolute claim in the abstract. Second, the 75% sentence-similarity filter may select easier sentences, and it is not clear whether M2M is evaluated on the same filtered subset as M2C/C2C. That could bias the comparison in C2C's favor. The authors need to report test-set sizes and, better, evaluate M2M on the aligned subset. Third, the automatic contour tracker's error is not reported; without it, the absolute mm error is hard to interpret. Minor issues: single speaker, no correction for multiple comparisons, no code/data release.

I would not block this on the central claim being false — it may well be true — but the evidence is one speaker and the absolute error is only meaningful relative to the same acquisition pipeline. The paper is worth a serious referee if the authors supply the missing details. For a reading group, it's a short, representative example of the field's current practice on MRI-based inversion.

My recommendation: send to peer review, but with a request for the tracker error, test-set overlap, and code/data release. Without those, the 1.56 mm is a feasibility signal, not a validated number.

Referee Report

4 major / 6 minor

Summary. The paper investigates acoustic-to-articulatory inversion from clean speech using a model trained on denoised MRI audio. Two corpora from the same French female speaker contain the same sentences: one recorded in an MRI scanner and denoised, the other recorded in a quiet room. The corpora are aligned via manually corrected phonetic segmentations and a phoneme-level temporal normalization procedure (Eqs. 2–5). A HuBERT-Bi-LSTM model is trained to predict eight articulator contours (50 points each) obtained by an automatic tracker, and compared under three conditions: MRI-to-MRI (M2M), MRI-trained on clean speech (M2C), and clean-to-clean (C2C). The paper reports mean RMSE values of 1.51, 1.64, and 1.56 mm, respectively, and a DTW-based alignment experiment showing worse performance. It concludes that clean speech supports articulatory inversion effectively and can enable real-world applications.

Significance. If the central claim is accepted, this would be a practical step toward using articulatory inversion outside the MRI scanner, avoiding the need for denoised scanner audio. The study has notable strengths: same speaker, identical sentence material, manually corrected phonetic segmentation, a held-out acquisition split, t-tests, and a DTW comparison. However, the absolute RMSE interpretation is undermined by the absence of a clean-speech articulatory ground truth, the unquantified error of the automatic contour tracker, and the acknowledged unmodeled Lombard/supine effects. The reported errors (1.5–1.6 mm) are also close to the MRI pixel size (1.62 mm), which limits the practical significance of sub-millimeter differences. The paper does not include code or data for reproducibility, further limiting verification.

major comments (4)

[§2, §4.1, Table 1] The central claim that clean speech inversion achieves an RMSE of 1.56 mm relies on the assumption that phoneme-level time-stretching (Eqs. 2–5) makes the MRI and clean recordings articulatorily equivalent. The paper explicitly acknowledges in §2 that the Lombard effect and supine posture are not modeled. Time-stretching only normalizes phoneme durations; it cannot remove articulatory differences caused by scanner noise and posture. Therefore, the MRI-derived contours are not a valid ground truth for the clean speech signal. The reported RMSE conflates inversion error with cross-condition production mismatch. The authors should either provide independent evidence that the two speaking conditions produce the same vocal-tract movements, or reframe the results as measuring cross-condition mapping consistency rather than clean-speech inversion accuracy.
[§3.1, §4.4, Table 1] The ground-truth contours are obtained from the automatic tracking method of [13], but the segmentation error of this tracker is not reported. Given that the MRI pixel size is 1.62 mm and the reported RMSE values are around 1.5–1.6 mm, the tracker's own error could be the same magnitude as the effects under study. Without a validation of the tracker accuracy against manual contours or some other reference, the absolute error interpretation is unsafe. The authors should report a segmentation-error estimate and discuss how it affects the RMSE comparisons.
[§4.5, §5, Tables 1–2] The statistical analysis uses t-tests on frame-level errors without accounting for the strong temporal dependence within utterances/sentences and across the eight articulators. No multiple-comparison correction is applied despite 8 articulators × 3 conditions (Table 1) and comparisons in Table 2. Moreover, the 75% sentence-similarity threshold in §4.1 may exclude difficult or variable sentences, biasing the results toward easier items. The authors should report the number/percentage of retained sentences, use per-sentence or per-acquisition error summaries with appropriate paired statistics, and apply a multiple-comparison correction or justify why it is unnecessary.
[§5, Table 2] The comparison with DTW-based alignment is valuable, but it also shows that the alignment method changes the RMSE by about 0.1 mm (1.64 vs 1.71 for M2C; 1.56 vs 1.68 for C2C). Combined with the unquantified production mismatch and tracker error, the small differences between M2M (1.51), C2C (1.56), and M2C (1.64) may be within the noise of the alignment/segmentation pipeline. The conclusion that clean speech is 'close to MRI-based performance' should be tempered, and the absolute error should not be presented as a validated measure of articulatory inversion accuracy without additional evidence.

minor comments (6)

[§5, Table 2] The text states that for C2C-DTW the RMSE ranges between 1.20 mm and 2.26 mm, but Table 2 shows a minimum of 1.17 mm (upper lip) and a maximum of 2.66 mm (tongue). Please correct the in-text ranges.
[Eqs. (2)–(5)] Equation (5) does not specify whether the frame index is rounded or floored, and the roles of f_s and Δt_frame should be clarified: f_s is the sampling frequency of the audio, while Δt_frame is the MRI frame period. The final index should presumably be rounded to an integer HuBERT frame; please state this explicitly.
[§3] The statement that 'both phonetic segmentations were carefully reviewed to ensure they were strictly identical across the two corpora' is unclear: the segmentations are time-aligned differently and come from different aligners, so 'strictly identical' cannot mean identical time boundaries. Please clarify what was verified (e.g., identical phoneme labels and sentence content).
[§4.4] The t-test is mentioned but not defined. Please specify whether it is paired or unpaired, and the unit of analysis (frames, sentences, or acquisitions). This is important for interpreting the statistical significance markers.
[Abstract / Conclusion] The phrase 'close to MRI-based performance' is used in the abstract, but the C2C mean RMSE (1.56) differs from M2M (1.51) and the individual articulator differences are not always consistent. A more cautious wording, such as 'comparable within the experimental uncertainty', would be more accurate given the caveats in the major comments.
[Supplementary material] Figure 1 references supplementary audio files, but these are not accessible in the manuscript PDF. Please ensure they are provided with the submission for review.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RMSE values are genuine held-out evaluations against MRI-derived contours, not fitted constants or renamed inputs.

full rationale

The paper's central derivation is a standard supervised inversion comparison: an acoustic-to-articulatory model is trained on denoised MRI speech and/or clean speech, and evaluated by RMSE against held-out MRI contours. The alignment procedure (Eqs. 2-5) explicitly constructs a target time in the clean corpus for each MRI frame, but the RMSE is computed against the actual MRI contours, which are independent of the model's predictions; there is no equation reducing the reported 1.56 mm to a fitted parameter or to the alignment itself. The self-citations to [13], [14], [15] supply the contour tracker and architecture, but these are published methods used as tools rather than the result being derived, and the paper does not invoke any uniqueness theorem from prior work. The main weakness - that MRI and clean recordings may not be articulatorily equivalent because Lombard effect and supine posture are explicitly unmodeled (Section 2) - is a validity threat concerning whether RMSE measures true clean-speech inversion accuracy, not a circularity in the derivation chain. The automatic contour tracker's unquantified error is similarly an external measurement concern. No step reduces the conclusion to its inputs by definition, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. Its central claim rests on the validity of the alignment between two recording conditions and the accuracy of tracked contours, not on fitted physical constants.

free parameters (3)

sentence similarity threshold = 0.75
Sentences below 75% Gestalt similarity are discarded in alignment (Section 4.1), which can bias the training/test set toward easier, more consistent utterances.
intra-phoneme epsilon = 0.001 s
Added in Eq. 2 to avoid division by zero; not central but a numerical choice.
model hyperparameters = 300 units, lr=0.001, batch=10, patience=10
Chosen without ablations shown; standard values, but affect reported performance.

assumptions (4)

domain assumption HuBERT-Base embeddings contain sufficient acoustic-articulatory information.
Used as input features; paper says Base beat Large but gives no comparative numbers (Section 3.1).
domain assumption Automatic contour tracking [13] provides accurate vocal-tract ground truth.
All RMSE values are computed against these tracked contours; segmentation error is not quantified.
domain assumption Identical sentences and phonetic segmentation imply equivalent articulatory targets in MRI and clean conditions.
Relies on same speaker, same sentences, and linear intra-phoneme time warping (Eqs. 2-5); Lombard and supine-posture effects are acknowledged but ignored (Section 2).
domain assumption Manual phonetic alignment is consistent across the two corpora.
Authors state both segmentations were manually corrected and 'strictly identical' (Section 3), but inter-annotator reliability is not reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model." pith.science (2026). https://pith.science/paper/DRGYJPX4

@misc{pith2026260311845,
  author       = {Pith},
  title        = {Pith review of: Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DRGYJPX4}},
  note         = {Machine review of arXiv:2603.11845}
}

read the original abstract

Articulatory acoustic inversion reconstructs vocal tract shapes from speech. Real-time magnetic resonance imaging (rt-MRI) allows simultaneous acquisition of both the acoustic speech signal and articulatory information. Besides the complexity of rt-MRI acquisition, the recorded audio is heavily corrupted by scanner noise and requires denoising to be usable. For practical use, it must be possible to invert speech recorded without MRI noise. In this study, we investigate the use of speech recorded in a clean acoustic environment as an alternative to denoised MRI speech. To this end we compare two signals from the same speaker with identical sentences which are aligned using phonetic segmentation. A model trained on denoised MRI speech is evaluated on both denoised MRI and clean speech. We also assess a model trained and tested only on clean speech. Results show that clean speech supports articulatory inversion effectively, achieving an RMSE of 1.56 mm, close to MRI-based performance.

Figures

Figures reproduced from arXiv: 2603.11845 by the authors.

**Figure 2.** Segmentation of articulators contour tracked in two images of the rt-MRI film: Arytenoid cartilage, Epiglottis, Lower lip, Pharyngeal wall, Soft palate midline, Tongue, Upper lip, Vocal folds 4. Methods 4.1. Alignment of MRI and Clean Speech Corpora It was necessary to align the MRI corpus with the clean speech corpus. After manually verifying the phonetic segmentations and ensuring their consistency, we developed a… view at source ↗

**Figure 1.** [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 3.** Model architecture 4.3. Loss function Articulatory acoustic inversion is a regression task. In this study, the objective is to minimize the distance between the predicted and reference articulatory points. We used the Mean Squared Error (MSE) loss function, which is the most commonly used metric for such regression problems. MSE = 1 n Xn i=1 (yi − yˆi) 2 (6) where n represents the total number of observations in th… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 2 linked inside Pith

[13]

A multi-channel/multi-speaker articulatory database for continuous speech recognition research,

A. Wrench, “A multi-channel/multi-speaker articulatory database for continuous speech recognition research,”5th Seminar on Speech Production: Models and Data, Univ. Saarland, Res. Rep., vol. 5, pp. 1–13, 01 2000

2000
[1]

Since the 1970s, several families of solutions have been pro- posed to address this complex inverse problem

Introduction Acoustic-to-articulatory inversion aims to estimate the geomet- ric shape of the vocal tract from the acoustic speech signal. Since the 1970s, several families of solutions have been pro- posed to address this complex inverse problem. With the availability of measured articulatory data, such as Electromagnetic Articulography (EMA) [1] and X-r...
[2]

How- ever, speech recorded in an MRI machine, even if it is denoised before use, is still far from speech produced in a quiet envi- ronment

Impact of MRI noise and denoising Our goal is to be able to use articulatory acoustic inversion in real-world applications, which means that the inversion must be able to use speech recorded in a quiet environment. How- ever, speech recorded in an MRI machine, even if it is denoised before use, is still far from speech produced in a quiet envi- ronment. F...
[3]

The first corpus was recorded at the Centre Hospitalier R ´egional Universitaire (CHRU) de Nancy and contains approximately 2.5 hours of data

Dataset We used two different corpus recorded by the same native French female speaker. The first corpus was recorded at the Centre Hospitalier R ´egional Universitaire (CHRU) de Nancy and contains approximately 2.5 hours of data. It consists of 105 acquisitions, each lasting 80 seconds and containing 4000 MRI frames. The rt-MRI images have a spatial reso...
[4]

Apr`es une heure

Methods 4.1. Alignment of MRI and Clean Speech Corpora It was necessary to align the MRI corpus with the clean speech corpus. After manually verifying the phonetic segmentations and ensuring their consistency, we developed a hierarchical alignment algorithm based on phonetic information. The alignment was performed hierarchically, proceeding from sentence...
[5]

The M2M configuration remains the best, with an average RMSE of 1.51 mm and a median of 1.33 mm

Results Table 1 compares the results obtained with the three configura- tions: M2M, M2C, and C2C. The M2M configuration remains the best, with an average RMSE of 1.51 mm and a median of 1.33 mm. It is followed by C2C, which achieved an average RMSE of 1.56 mm and a median of 1.33 mm. Finally, the M2C model obtained an average RMSE of 1.64 mm and a median ...
[6]

This configuration corresponds to that of our previous work on in- version, using denoised speech for both training and testing

Discussion The results shown in Table 1 indicate that M2M configuration achieves the best scores, both globally and per articulator. This configuration corresponds to that of our previous work on in- version, using denoised speech for both training and testing. There is therefore no mismatch between the training and test data, which explains why this conf...
[7]

Input for training and evaluation is MRI speech (M2M)

Show all 40 references

[8]

Input for training is denoised MRI speech, and for evaluation is clean speech (M2C)
[9]

Input for training and evaluation is clean speech (C2C). For all configurations involving clean speech, the signals Table 1:Comparison of RMSE (mm) and Median (mm) between the M2M, M2C, and C2C conditions M2M M2C C2C RMSE Median RMSE Median RMSE Median Arytenoid cartilage 1.72...
[10]

Speaker dependent acoustic-to-articulatory inversion using real-time MRI of the vocal tract,

T. Csap ´o, “Speaker dependent acoustic-to-articulatory inversion using real-time MRI of the vocal tract,” inProc. INTERSPEECH 2020 – 21st Annual Conference of the International Speech Com- munication Association, Shanghai, 10 2020, pp. 3720–3724

2020
[11]

Preprocessing for acoustic-to-articulatory inversion using real-time mri movies of japanese speech,

A. Oura, H. Kikuchi, and T. Kobayashi, “Preprocessing for acoustic-to-articulatory inversion using real-time mri movies of japanese speech,” inProceedings of 25th Conference Interspeech, 09 2024, pp. 1550–1554

2024
[12]

Conclusion In this paper, we studied the possibility of reconstructing the shape of the vocal tract from clean speech rather than speech recorded in an MRI scanner and then denoised. Although the results obtained are slightly inferior to the reference system trained and tested...
[14]

X-ray mi- crobeam speech production database,

J. Westbury, P. Milenkovic, G. Weismer, and R. Kent, “X-ray mi- crobeam speech production database,”The Journal of the Acous- tical Society of America, vol. 88, no. S1, pp. S56–S56, 1990

1990
[15]

Acoustic-to-articulatory inversion mapping with gaussian mixture model

T. Toda, A. W. Black, and K. Tokuda, “Acoustic-to-articulatory inversion mapping with gaussian mixture model.” inINTER- SPEECH, 2004, pp. 1129–1132

2004
[16]

Estimation of articulatory movements from speech acoustics using an hmm-based speech production model,

S. Hiroya and M. Honda, “Estimation of articulatory movements from speech acoustics using an hmm-based speech production model,”IEEE Transactions on Speech and Audio Processing, vol. 12, no. 2, pp. 175–185, 2004

2004
[17]

A deep recurrent approach for acoustic-to-articulatory inversion,

P. Liu, Q. Yu, Z. Wu, S. Kang, H. Meng, and L. Cai, “A deep recurrent approach for acoustic-to-articulatory inversion,” in2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015, pp. 4450–4454

2015
[18]

Independent and Automatic Evaluation of Speaker-Independent Acoustic-to- Articulatory Reconstruction,

M. Parrot, J. Millet, and E. Dunbar, “Independent and Automatic Evaluation of Speaker-Independent Acoustic-to- Articulatory Reconstruction,” inProceedings of INTERSPEECH 2020, 21st Annual Conference of the International Speech Communication Association, ser. Proceedings of INT...

2020
[19]

Speaker-independent acoustic-to-articulatory speech inversion,

P. Wu, L.-W. Chen, C. J. Cho, S. Watanabe, L. Goldstein, A. W. Black, and G. K. Anumanchipalli, “Speaker-independent acoustic-to-articulatory speech inversion,” inIEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

2023
[20]

The secret source: In- corporating source features to improve acoustic-to-articulatory speech inversion,

Y . M. Siriwardena and C. Espy-Wilson, “The secret source: In- corporating source features to improve acoustic-to-articulatory speech inversion,” inIEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

2023
[21]

Analysis of speech production real-time mri,

V . Ramanarayanan, S. Tilsen, M. Proctor, J. T ¨oger, L. Goldstein, K. S. Nayak, and S. Narayanan, “Analysis of speech production real-time mri,”Computer Speech & Language, vol. 52, pp. 1–22, 2018

2018
[22]

Multimodal dataset of real-time 2D and static 3D MRI of healthy French speakers,

K. Isaieva, Y . Laprie, J. Lecl `ere, I. K. Douros, J. Felblinger, and P.-A. Vuissoz, “Multimodal dataset of real-time 2D and static 3D MRI of healthy French speakers,”Scientific Data , vol. 8, no. 1, p. 258, Oct. 2021. [Online]. Available: https://hal.science/hal-03507532

2021
[23]

and the supine position. We do not take this effect into arXiv:2603.11845v1 [eess.AS] 12 Mar 2026 account because we do not have the data to address this issue, as it is currently impossible to have an MRI machine in a vertical position

2026 arXiv
[24]

Speech2rtmri: Speech-guided diffusion model for real-time mri video of the vocal tract during speech,

H. Nguyen, S. Foley, K. Huang, X. Shi, T. Feng, and S. Narayanan, “Speech2rtmri: Speech-guided diffusion model for real-time mri video of the vocal tract during speech,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE,...

2025
[25]

Automatic segmentation of vocal tract articulators in real-time magnetic resonance imaging,

V . Ribeiro, K. Isaieva, J. Leclere, J. Felblinger, P.-A. Vuissoz, and Y . Laprie, “Automatic segmentation of vocal tract articulators in real-time magnetic resonance imaging,”Computer Methods and Programs in Biomedicine, vol. 243, no. 2, p. 107907, Jan. 2024. [Online]. Availa...

2024
[26]

Both phonetic segmentations were carefully reviewed to ensure they were strictly identical across the two corpora

and subsequently manually corrected by an expert. Both phonetic segmentations were carefully reviewed to ensure they were strictly identical across the two corpora. 3.1. Pre processing We compared the HuBERT Base and HuBERT Large models, and the best results were obtained with...
[27]

Complete Reconstruc- tion of the Tongue Contour Through Acoustic to Articulatory Inversion Using Real-Time MRI Data,

S. Azzouz, P.-A. Vuissoz, and Y . Laprie, “Complete Reconstruc- tion of the Tongue Contour Through Acoustic to Articulatory Inversion Using Real-Time MRI Data,” inICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Hyderabad, I...

2025
[28]

Reconstruction of the Complete V ocal Tract Contour Through Acoustic to Articulatory Inversion Using Real-Time MRI Data,

——, “Reconstruction of the Complete V ocal Tract Contour Through Acoustic to Articulatory Inversion Using Real-Time MRI Data,” inInterspeech 2025. Rotterdam (NL), Netherlands: ISCA, Aug. 2025, pp. 978–982. [Online]. Available: https: //hal.science/hal-05293831

2025
[29]

Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021

2021
[30]

Self-supervised models of speech infer universal articulatory kinematics,

C. J. Cho, A. Mohamed, A. W. Black, and G. K. Anumanchipalli, “Self-supervised models of speech infer universal articulatory kinematics,” inICASSP 2024-2024 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 12 061–12 065

2024
[31]

wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,

A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,”Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020

2020
[32]

Wavlm: Large-scale self- supervised pre-training for full stack speech processing,

S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self- supervised pre-training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

2022
[33]

Improv- ing speech inversion through self-supervised embeddings and en- hanced tract variables,

A. A. Attia, Y . M. Siriwardena, and C. Espy-Wilson, “Improv- ing speech inversion through self-supervised embeddings and en- hanced tract variables,” in2024 32nd European Signal Processing Conference (EUSIPCO). IEEE, 2024, pp. 306–310

2024
[34]

A fine-tuned wav2vec 2.0/hubert benchmark for speech emotion recognition, speaker verification and spoken language understanding,

Y . Wang, A. Boumadane, and A. Heba, “A fine-tuned wav2vec 2.0/hubert benchmark for speech emotion recognition, speaker verification and spoken language understanding,”arXiv preprint arXiv:2111.02735, 2021

2021 arXiv
[35]

The influence of acoustics on speech production: A noise-induced stress phenomenon known as the lombard reflex,

J.-C. Junqua, “The influence of acoustics on speech production: A noise-induced stress phenomenon known as the lombard reflex,” Speech communication, vol. 20, no. 1-2, pp. 13–22, 1996

1996
[36]

A general flexible frame- work for the handling of prior information in audio source sepa- ration,

A. Ozerov, E. Vincent, and F. Bimbot, “A general flexible frame- work for the handling of prior information in audio source sepa- ration,”IEEE Transactions on Audio Speech and Language Pro- cessing, vol. 20, 05 2012

2012
[37]

De l’importance de l’homog ´en´eisation des conventions de transcription pour l’alignement automatique de corpus oraux de parole spontan ´ee,

D. Fohr, O. Mella, and D. Jouvet, “De l’importance de l’homog ´en´eisation des conventions de transcription pour l’alignement automatique de corpus oraux de parole spontan ´ee,” in8es Journ ´ees Internationales de Linguistique de Corpus (JLC2015), 2015

2015
[38]

Montreal forced aligner: Trainable text-speech align- ment using kaldi

M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Son- deregger, “Montreal forced aligner: Trainable text-speech align- ment using kaldi.” inInterspeech, vol. 2017, 2017, pp. 498–502

2017
[39]

Pattern matching: The gestalt approach,

J. W. Ratcliff, D. E. Metzeneret al., “Pattern matching: The gestalt approach,”Dr. Dobb’s Journal, vol. 13, no. 7, p. 46, 1988

1988
[40]

M ¨uller,Fundamentals of music processing: Audio, analysis, algorithms, applications

M. M ¨uller,Fundamentals of music processing: Audio, analysis, algorithms, applications. Springer, 2015, vol. 5

2015

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [13]

A multi-channel/multi-speaker articulatory database for continuous speech recognition research,

A. Wrench, “A multi-channel/multi-speaker articulatory database for continuous speech recognition research,”5th Seminar on Speech Production: Models and Data, Univ. Saarland, Res. Rep., vol. 5, pp. 1–13, 01 2000

2000

[2] [1]

Since the 1970s, several families of solutions have been pro- posed to address this complex inverse problem

Introduction Acoustic-to-articulatory inversion aims to estimate the geomet- ric shape of the vocal tract from the acoustic speech signal. Since the 1970s, several families of solutions have been pro- posed to address this complex inverse problem. With the availability of measured articulatory data, such as Electromagnetic Articulography (EMA) [1] and X-r...

[3] [2]

How- ever, speech recorded in an MRI machine, even if it is denoised before use, is still far from speech produced in a quiet envi- ronment

Impact of MRI noise and denoising Our goal is to be able to use articulatory acoustic inversion in real-world applications, which means that the inversion must be able to use speech recorded in a quiet environment. How- ever, speech recorded in an MRI machine, even if it is denoised before use, is still far from speech produced in a quiet envi- ronment. F...

[4] [3]

The first corpus was recorded at the Centre Hospitalier R ´egional Universitaire (CHRU) de Nancy and contains approximately 2.5 hours of data

Dataset We used two different corpus recorded by the same native French female speaker. The first corpus was recorded at the Centre Hospitalier R ´egional Universitaire (CHRU) de Nancy and contains approximately 2.5 hours of data. It consists of 105 acquisitions, each lasting 80 seconds and containing 4000 MRI frames. The rt-MRI images have a spatial reso...

[5] [4]

Apr`es une heure

Methods 4.1. Alignment of MRI and Clean Speech Corpora It was necessary to align the MRI corpus with the clean speech corpus. After manually verifying the phonetic segmentations and ensuring their consistency, we developed a hierarchical alignment algorithm based on phonetic information. The alignment was performed hierarchically, proceeding from sentence...

[6] [5]

The M2M configuration remains the best, with an average RMSE of 1.51 mm and a median of 1.33 mm

Results Table 1 compares the results obtained with the three configura- tions: M2M, M2C, and C2C. The M2M configuration remains the best, with an average RMSE of 1.51 mm and a median of 1.33 mm. It is followed by C2C, which achieved an average RMSE of 1.56 mm and a median of 1.33 mm. Finally, the M2C model obtained an average RMSE of 1.64 mm and a median ...

[7] [6]

This configuration corresponds to that of our previous work on in- version, using denoised speech for both training and testing

Discussion The results shown in Table 1 indicate that M2M configuration achieves the best scores, both globally and per articulator. This configuration corresponds to that of our previous work on in- version, using denoised speech for both training and testing. There is therefore no mismatch between the training and test data, which explains why this conf...

[8] [7]

Input for training and evaluation is MRI speech (M2M)