PRiSM: Benchmarking Phone Realization in Speech Models

Brendon Boldt; Chao-Han Huck Yang; Chin-Jou Li; Darsh Agrawal; David R. Mortensen; Eunjung Yeo; Hanyu Zhou; Jian Zhu; Kalvin Chang; Karen Rosero Jacome

REVIEW 3 major objections 4 minor 3 cited by

This paper argues that phone recognition systems should be benchmarked not only on transcription error but also on downstream utility, and reports that diverse language exposure, encoder-CTC architectures, and specialized PR models—not larg

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 09:17 UTC pith:BGRB4BPD

load-bearing objection PRiSM is a genuinely useful benchmark with broad evaluation and open code; the main soft spot is the unsecured 'unseen language' claim for LALMs, but the paper already half-admits it and the core findings still hold. the 3 major comments →

arxiv 2601.14046 v2 pith:BGRB4BPD submitted 2026-01-20 cs.CL cs.SD

PRiSM: Benchmarking Phone Realization in Speech Models

Shikhar Bharadwaj , Chin-Jou Li , Yoonjae Kim , Kwanghee Choi , Eunjung Yeo , Ryan Soh-Eun Shim , Hanyu Zhou , Brendon Boldt

show 8 more authors

Karen Rosero Jacome Kalvin Chang Darsh Agrawal Keer Xu Chao-Han Huck Yang Jian Zhu Shinji Watanabe David R. Mortensen

This is my paper

classification cs.CL cs.SD

keywords phone recognitionphonetic feature error ratebenchmarkmultilingual speech modelslarge audio language modelsrepresentation probingextrinsic evaluationlow-resource languages

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Phone recognition (PR) maps speech to phonetic units and is the atomic interface for cross-lingual speech processing, but standard evaluations only measure surface transcription accuracy. PRiSM is introduced as the first open-source benchmark that pairs intrinsic transcription evaluation with extrinsic downstream probes—pathological speech assessment, L2 speech evaluation, and multilingual identification—using both predicted transcripts and hidden representations. The paper's empirical findings are that broad and diverse language exposure during training is the key driver of PR performance, encoder-CTC models are the most stable across domains, and specialized PR models still outperform large audio-language models. A sympathetic reader would care because PRiSM gives the field a common, reproducible yardstick for judging whether models truly perceive phonetic detail rather than merely guessing familiar patterns.

Core claim

PRiSM's central claim is that phone recognition systems need to be evaluated on two channels: transcription accuracy and downstream utility, because error rates alone do not reveal phonetic capability or practical value. The benchmark measures intrinsic ability with a phonetic-feature edit distance (PFER) on both seen-variation and unseen-language datasets, and extrinsic utility by feeding predicted transcriptions and internal representations into lightweight probes for clinical, educational, and multilingual tasks. Across those probes, the paper reports that language diversity in training data matters more than raw scale, encoder-CTC architectures provide the most stable performance, and la

What carries the argument

PRiSM rests on a two-channel evaluation scaffold. Intrinsically, PFER computes edit distance in articulatory-feature space rather than over raw phone symbols, so errors are linguistically meaningful. Extrinsically, each system is probed twice: a transcript probe passes predicted IPA through a small bidirectional GRU for the target task, while a representation probe pools last-layer hidden states with attention and an MLP. The benchmark also includes a masked-phone diagnostic that silences a percentage of speech and measures PFER, separating models that listen to the acoustic signal from models that lean on phonotactic guessing.

Load-bearing premise

The unseen-language results assume the benchmark languages never appeared in any evaluated model's training data; the paper itself notes that some may have appeared in LALM training.

What would settle it

Check the training corpora used for the two large audio-language models for the 45 DoReCo languages, the 95 VoxAngeles languages, and Tusom; if any appear, re-run the unseen-language evaluations on languages verified to be absent and see whether the reported gap closes.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If correct, future PR evaluations should include both intrinsic transcription metrics and downstream probes; intrinsic error alone misses task-dependent strengths.
Language diversity in pretraining and supervised fine-tuning drives generalization to unseen languages, so collecting phonetically transcribed data from more languages may matter more than adding hours of data in familiar languages.
Encoder-only CTC-based architectures offer the most stable phonetic performance across new domains, making them a safer default for multilingual model development.
Specialized PR models currently outperform large audio-language models on phone recognition and most downstream phonetic tasks, so practitioners should not assume LALMs are ready for clinical, educational, or low-resource use.
Phonotactic reliance is measurable through masking, and models that preserve acoustic fidelity at high masking rates are more trustworthy for transcribing atypical or dialectal speech.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the 'unseen' benchmark languages turn out to have appeared in LALM training data, the reported LALM gap would shrink; a verified held-out language set would make the comparison decisive.
The transcript-probe versus representation-probe trade-off suggests the two channels measure complementary abilities, so clinical or educational systems may want to select models based on the specific probe rather than a single aggregate score.
The masked-phone experiment could be standardized as a standalone 'acoustic reliance' index, giving developers a cheap diagnostic before deploying a model on low-resource dialects.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

PRiSM is a genuinely useful benchmark with broad evaluation and open code; the main soft spot is the unsecured 'unseen language' claim for LALMs, but the paper already half-admits it and the core findings still hold.

read the letter

PRiSM is a solid, useful benchmark paper. The strongest part is the breadth: nine models, thirteen tasks, intrinsic PFER plus extrinsic transcript and representation probes, and a genuinely new packaged comparison. The curated Vaani-Hi subset and the UltraSuite mismatch corrections are real contributions, and the code/release plan makes this a plausible community resource. The findings are directionally consistent across datasets: diverse language coverage helps, encoder-CTC models are stable, LALMs lag on phonetic transcription. That consistency is the best evidence the paper has, and it's decent evidence.

What it does well beyond the benchmark itself: the analyses in Section 6 are thoughtful. The phone-masking experiment cleanly separates acoustic fidelity from phonotactic reliance, the inventory-induction results reward multilingual training in a controlled way, and the geolocation attribution example is a nice touch. The limitations section is unusually honest — transcription is not ground truth, probes can overfit spurious cues, and default prompts may understate LALMs.

The soft spots are real but not fatal. First, intrinsic PFER numbers have no error bars or significance tests. For a benchmark that will be used to rank models, that's a gap. Second, the LALMs are only evaluated zero-shot, so they aren't compared on the same footing as the specialized models on extrinsic tasks. Third, the aggregate score's task selection (K=6) looks post hoc — the excluded tasks (LID-fl, GEO-v, PI-drc) are exactly the ones where specialized models struggle relative to Whisper, and the included ones favor the paper's narrative. That choice should be justified or rerun with all tasks.

The 'unseen language' concern in the stress-test note is the most important. The paper's own footnote says some unseen languages may have appeared in LALM training data, and Section 6.2 asserts DoReCo absence from all model training without verification. For closed LALMs like Gemini that claim is unprovable, and for XLSR-53 it's not checked either. So the clean 'generalization' story for the unseen-language rows is not fully secured. That said, the LALM lag also shows up on seen-language variation and downstream tasks, so the main conclusion doesn't collapse.

Who is this for? Speech and phonetics researchers who want a standard evaluation suite, and practitioners choosing between PR models. It deserves a serious referee — with requests for uncertainty estimates, a fuller aggregate-score analysis, and a softer or better-verified unseen-language claim.

I'd bring it to a reading group and cite it. Send to peer review, conditional on those revisions.

Referee Report

3 major / 4 minor

Summary. The paper introduces PRiSM, an open-source benchmark for evaluating phone recognition (PR) systems. PRiSM combines intrinsic evaluation via Phonetic Feature Error Rate (PFER) on seen-language variation (TIMIT, L2-ARCTIC, Speech Accent Archive) and unseen languages (DoReCo, VoxAngeles, Tusom2021) with extrinsic evaluation of downstream utility in pathological, L2, and multilingual speech tasks, using both transcript probes and representation probes. The authors evaluate specialized PR models (Wav2Vec2-based, ZIPA, POWSM, POWSM-CTC) and two LALMs (Gemini 2.5 Flash, Qwen3-Omni-Instruct), and report that multilingual language coverage is key, encoder-CTC models are the most stable, and LALMs lag behind specialized PR systems. The benchmark code, recipes, and datasets are released.

Significance. If the results are secured, PRiSM would be a valuable community resource: it provides a standardized PFER metric, a broad multilingual testbed, reproducible evaluation recipes, and released datasets. The inclusion of both intrinsic and extrinsic views, and the separation of transcript vs. representation probes, is a useful step beyond single-metric PR evaluations. However, the headline claims about unseen-language generalization and LALM performance rest on assumptions that the paper itself partially concedes are unverified, and the extrinsic LALM comparison uses a different protocol from the other models. These issues are fixable but currently weaken the central conclusions.

major comments (3)

[§5.1, Table 3 footnote, §6.2] The 'unseen-language' evaluation is not secured for LALMs. The footnote to Table 3 explicitly says that 'some of the “unseen languages” may have appeared in the training data' for Gemini 2.5 Flash and Qwen3-Omni-Instruct, and §5.1 repeats this caveat. Yet §6.2 states that all DoReCo languages are 'absent from the training corpora of all models,' which is internally inconsistent for those two models. Since PR-drc, PR-vox, and PR-tsm are the basis for the claim that unseen languages benefit from multilingual training and for the LALM-lag conclusion, the comparison is not a clean generalization test for LALMs. The authors should either verify non-inclusion via contamination checks or explicitly restrict the generalization claim to models whose training data are known, reporting LALM results on these sets only as seen-data behavior.
[§4, Table 4] The extrinsic comparison between LALMs and other systems is not controlled. LALMs are evaluated with zero-shot prompting, while all other models are evaluated with trained transcript probes (bi-GRU) or representation probes (attention pooling + MLP). Thus the downstream conclusion that 'LALMs remain limited' (and the aggregate Score column in Table 4) confounds model capability with evaluation interface. For instance, Qwen3-Omni's zero-shot scores on pathological tasks are not directly comparable to the TP/RP scores of other models. The authors should either probe LALM representations or transcripts with the same trained probes, or explicitly treat zero-shot prompting as a separate protocol and avoid direct rankings across protocols.
[Table 3, §5.1] No uncertainty or significance testing is provided for the intrinsic PFER values, although several supporting claims hinge on small differences between models. For example, PR-tmt values cluster at 13.1–13.7 and PR-arc at 9.7–10.3 for multiple systems. Without bootstrap confidence intervals or significance tests over test utterances, the claim that 'encoder-CTC models are the most stable' and fine-grained rankings (e.g., ZIPA-CTC-NS vs. POWSM-CTC on PR-arc) are not established. PFER is deterministic given a model and test set, but the finite test-sample noise still matters; the authors should quantify it.

minor comments (4)

[Table 1, Table 5] The dataset name appears as 'V oxAngeles' in Table 1 and Table 5, but as 'VoxAngeles' in the text (§5.1). Please standardize.
[§C] Typo: 'Kentall Tau' should be 'Kendall Tau'.
[§B.2, Eq. (2)] The aggregate Score definition uses K=6 and explicitly excludes LID-fl, GEO-v, and PI-drc, with the rationale that those tasks 'show differentiation.' This selection is post hoc and should be justified more transparently; otherwise the Score is not a full summary of the benchmark's extrinsic tasks.
[§6.3] The claim that transcript probes 'significantly outperform' representation probes on GEO-v is not supported by any significance test; the mean error difference is reported, but no error bars or tests are given for this comparison.

Circularity Check

0 steps flagged

No significant circularity; PRiSM is an empirical benchmark against public datasets with no derivation chain that reduces to its inputs.

full rationale

PRiSM is a benchmark paper, not a derivation: it defines an evaluation framework (PFER over PanPhon features, transcript/representation probes), applies it to existing and described PR systems, and reports measured performance on public datasets. There is no claimed first-principles result whose derivation could be equivalent to its inputs. The PFER metric is defined externally via PanPhon (Mortensen et al., 2016) and edit distance over articulatory features; it is not fitted to any model's outputs. The evaluated systems include models from the authors' prior work (ZIPA, POWSM), but these are benchmarked objects, not load-bearing citations used to justify the framework's validity or to forbid alternatives. The paper's own footnote in Table 3 concedes that 'some of the “unseen languages” may have appeared in the training data' for LALMs; this is a data-contamination risk that weakens the generalization claim, but it is not circularity, since the measurements are still independent of the benchmark's construction. No fitted parameter is relabeled as a prediction, no uniqueness theorem is imported from the authors, and no ansatz is smuggled in via self-citation. The central findings come from comparisons on fixed, externally sourced corpora, so the paper is self-contained against external benchmarks. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The benchmark introduces no new theoretical entities. The main load-bearing assumptions are about evaluation validity: the feature-based error metric, the 'unseen' status of certain datasets, the fairness of probe architectures, and the acoustic-masking probe. The hand-picked aggregation weighting and task selection are the closest to free parameters, but they are evaluation design choices rather than fitted constants.

free parameters (2)

Aggregate score log-weighting (w_i = ln N_i) = ln(N_i)
Hand-picked to dampen test-set size disparity in the PRiSM extrinsic score; affects the ranking of models and is not derived from a statistical model.
Number of tasks in aggregate score (K=6) = 6
Selected post hoc as tasks that show differentiation in model behavior; arbitrary and could bias the overall ranking.

axioms (4)

domain assumption PFER computed via PanPhon feature edit distance with equal feature weights is a valid measure of phonetic error.
Used in Eq. 1 for all intrinsic evaluations; assumes all articulatory features are equally important.
domain assumption DoReCo, VoxAngeles, and Tusom are outside the training data of all evaluated models.
Underlies the 'unseen languages' claims; the paper itself footnotes that LALMs may have seen these languages in training (§5.1).
domain assumption The transcript probe (bi-GRU) and representation probe (attention+MLP) are fair and sufficient comparators of downstream utility.
Used in all extrinsic evaluations; probe architecture choices may favor certain models (e.g., RNN preserves order for geolocation TP).
domain assumption Phone masking with silence measures whether a model relies on acoustics vs. phonotactics.
Assumed in §6.1 and Figure 2; a horizontal line is interpreted as pure acoustic reliance, but silence can also trigger decoder-specific behaviors.

pith-pipeline@v1.3.0-alltime-deepseek · 22927 in / 8126 out tokens · 86980 ms · 2026-08-03T09:17:07.064728+00:00 · methodology

0 comments

read the original abstract

Phone recognition (PR) serves as the atomic interface for language-agnostic modeling for cross-lingual speech processing and phonetic analysis. Despite prolonged efforts in developing PR systems, current evaluations only measure surface-level transcription accuracy. We introduce PRiSM, the first open-source benchmark designed to expose blind spots in phonetic perception through intrinsic and extrinsic evaluation of PR systems. PRiSM standardizes transcription-based evaluation and assesses downstream utility in clinical, educational, and multilingual settings with transcription and representation probes. We find that diverse language exposure during training is key to PR performance, encoder-CTC models are the most stable, and specialized PR models still outperform Large Audio Language Models. PRiSM releases code, recipes, and datasets to move the field toward multilingual speech models with robust phonetic ability: https://github.com/changelinglab/prism.

Figures

Figures reproduced from arXiv: 2601.14046 by Brendon Boldt, Chao-Han Huck Yang, Chin-Jou Li, Darsh Agrawal, David R. Mortensen, Eunjung Yeo, Hanyu Zhou, Jian Zhu, Kalvin Chang, Karen Rosero Jacome, Keer Xu, Kwanghee Choi, Ryan Soh-Eun Shim, Shikhar Bharadwaj, Shinji Watanabe, Yoonjae Kim.

**Figure 2.** Figure 2: PFER vs Phone masking rate. A PR model that relies only on acoustics should produce a horizontal line. Encoder-only models trained with CTC loss retain acoustic fidelity at high masking levels. See § 6.1. phone distribution differences across regions. Finally, in § 6.4, we assess zero-shot performance of LALMs on challenging tasks, concluding that they remain insensitive to sociophonetic variation. 6.1 … view at source ↗

**Figure 3.** Figure 3: Precision and Recall scores of PR systems [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

**Figure 4.** Figure 4: Attribution map from Vaani (Ghosh et al., 2025). Red supports and blue opposes correct geolocation. W2V2P-LV60 detects doubled phones (§ 6.3). West) and 1703 km (North-South), covering the entire Hindi speaking region of India. This performance is surprising, as the cascade-based approach loses suprasegmental information such as intonation that provide strong phonetic cues for the differentiation of dia… view at source ↗

**Figure 5.** Figure 5: Normalized confusion matrices for Gemini [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗

discussion (0)

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

An Empirical Recipe for Universal Phone Recognition
cs.CL 2026-03 conditional novelty 6.0

Finetuning the multilingual SSL model XEUS with Self-Conditioned CTC on 17k hours of IPAPack++ yields SOTA phone recognition on 100+ languages and accented English.
MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition
cs.CL 2026-07 conditional novelty 5.0

Grouping 495 languages into roughly 16 clusters and routing speech to group-specific LoRA experts improves multilingual ASR error rates over dense and random baselines.
BranchShine: Compact Raw-Audio-to-IPA Transcription with a RoPE E-Branchformer Encoder
cs.LG 2026-06 unverdicted novelty 4.0

A 33M-parameter raw-audio CTC model with 19-block RoPE E-Branchformer achieves 9.19% whitespace-insensitive IPA CER on a 16,660-utterance 41-language test set, outperforming a 575M-parameter PhoneticXEUS baseline at 9...

Reference graph

Works this paper leans on

12 extracted references · 4 linked inside Pith · cited by 3 Pith papers

[5]

11 Tiago Pimentel, Brian Roark, and Ryan Cotterell

When do prompting and prefix-tuning work? a theory of capabilities and limitations.arXiv preprint arXiv:2310.19698. 11 Tiago Pimentel, Brian Roark, and Ryan Cotterell

Pith/arXiv arXiv
[6]

Transactions of the Association for Computational Linguistics, 8:1–18

Phonotactic complexity and its trade-offs. Transactions of the Association for Computational Linguistics, 8:1–18. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brock- man, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak su- pervision. InInternational conference on machine learning, pages 28492–28518. PMLR. Karen Ro...

arXiv 2023
[7]

InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 1–5

The edinburgh international accents of en- glish corpus: Towards the democratization of en- glish asr. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 1–5. IEEE. Tanja Schultz and Katrin Kirchhoff. 2006.Multilin- gual speech processing. Elsevier. Xian Shi, Fan Yu, Yizhou Lu, Yuhao Liang, Qiangz...

2023
[12]

‘json), and do not include any conversational filler. • Field“transcription“: –Must contain EXACTLY ONE string sequence of IPA symbols. –No slashes/or brackets[]. –Example:

is a dataset of 50+ small or endangered languages with broad phonetic transcriptions; we use the same DoReCo subset as Zhu et al. (2025, 2024). V oxAngeles (Chodroff et al., 2024) is a cleaned, 95-language version of the UCLA Pho- netics Lab Archive (Ladefoged et al., 2009). Tu- som2021 (Mortensen et al., 2021) is a dataset of speech and narrow phonetic t...

2025
[1892]

Patrick Foley, Matthew Wiesner, Bismarck Odoom, Leibny Paola Garcia Perera, Kenton Murray, and Philipp Koehn

ISCA. Patrick Foley, Matthew Wiesner, Bismarck Odoom, Leibny Paola Garcia Perera, Kenton Murray, and Philipp Koehn. 2024. Where are you from? ge- olocating speech and applications to language iden- tification. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (V...

2024
[2016]

InProceedings of COLING 2016, the 26th International Confer- ence on Computational Linguistics: Technical Pa- pers, pages 3475–3484

Panphon: A resource for mapping IPA seg- ments to articulatory feature vectors. InProceedings of COLING 2016, the 26th International Confer- ence on Computational Linguistics: Technical Pa- pers, pages 3475–3484. ACL. David R Mortensen, Jordan Picone, Xinjian Li, and Kathleen Siminyu. 2021. Tusom2021: A phonet- ically transcribed speech dataset from an en...

2016
[2017]

InInternational conference on machine learning, pages 3319–3328

Axiomatic attribution for deep networks. InInternational conference on machine learning, pages 3319–3328. PMLR. Ashima Suvarna, Harshita Khandelwal, and Nanyun Peng. 2024. Phonologybench: Evaluating phono- logical skills of large language models. InProceed- ings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 1–14. Chihi...

2024
[2018]

Investigating the role of l1 in automatic pro- nunciation evaluation of l2 speech.arXiv preprint arXiv:1807.01738. Joseph Turian, Jordie Shier, Humair Raj Khan, Bhik- sha Raj, Björn W Schuller, Christian J Stein- metz, Colin Malloy, George Tzanetakis, Gissel Ve- larde, Kirk McNally, and 1 others. 2022. Hear: Holistic evaluation of audio representations. I...

Pith/arXiv arXiv 2022
[2020]

InNeuRIPS Workshop on Self- Supervised Learning for Speech and Audio Process- ing

The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken lan- guage modeling. InNeuRIPS Workshop on Self- Supervised Learning for Speech and Audio Process- ing. Ludger Paschen, François Delafontaine, Christoph Draxler, Susanne Fuchs, Matthew Stave, and Frank Seifart. 2020. Building a time-aligned cross- linguistic reference c...

arXiv 2021
[2021]

InIEEE International Conference on Acoustics, Speech, and Signal Processing

The accented english speech recognition chal- lenge 2020: Open datasets, tracks, baselines, results and methods. InIEEE International Conference on Acoustics, Speech, and Signal Processing. Ryan Soh-Eun Shim, Kalvin Chang, and David R Mortensen. 2024. Phonotactic complexity across dialects. InProceedings of the 2024 Joint In- ternational Conference on Com...

2020
[2022]

Please call Stella

Simple and effective zero-shot cross-lingual phoneme recognition. InInterspeech 2022, pages 2113–2117. 12 Wei Xue, Roeland van Hout, Catia Cucchiarini, and Helmer Strik. 2023. Assessing speech intelligibility of pathological speech in sentences and word lists: The contribution of phoneme-level measures.Jour- nal of Communication Disorders, 102:106301. Chi...

Pith/arXiv arXiv 2022
[2023]

International Phonetic Association

Assessment of l2 intelligibility: Comparing l1 listeners and automatic speech recognition.Re- CALL: the Journal of EUROCALL, 35(1):89–104. International Phonetic Association. 1999.Handbook of the International Phonetic Association: A guide to the use of the International Phonetic Alphabet. Cambridge University Press. Heejin Kim, Mark Hasegawa-Johnson, Adr...

Pith/arXiv arXiv 1999

[1] [5]

11 Tiago Pimentel, Brian Roark, and Ryan Cotterell

When do prompting and prefix-tuning work? a theory of capabilities and limitations.arXiv preprint arXiv:2310.19698. 11 Tiago Pimentel, Brian Roark, and Ryan Cotterell

Pith/arXiv arXiv

[2] [6]

Transactions of the Association for Computational Linguistics, 8:1–18

Phonotactic complexity and its trade-offs. Transactions of the Association for Computational Linguistics, 8:1–18. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brock- man, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak su- pervision. InInternational conference on machine learning, pages 28492–28518. PMLR. Karen Ro...

arXiv 2023

[3] [7]

InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 1–5

The edinburgh international accents of en- glish corpus: Towards the democratization of en- glish asr. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 1–5. IEEE. Tanja Schultz and Katrin Kirchhoff. 2006.Multilin- gual speech processing. Elsevier. Xian Shi, Fan Yu, Yizhou Lu, Yuhao Liang, Qiangz...

2023

[4] [12]

‘json), and do not include any conversational filler. • Field“transcription“: –Must contain EXACTLY ONE string sequence of IPA symbols. –No slashes/or brackets[]. –Example:

is a dataset of 50+ small or endangered languages with broad phonetic transcriptions; we use the same DoReCo subset as Zhu et al. (2025, 2024). V oxAngeles (Chodroff et al., 2024) is a cleaned, 95-language version of the UCLA Pho- netics Lab Archive (Ladefoged et al., 2009). Tu- som2021 (Mortensen et al., 2021) is a dataset of speech and narrow phonetic t...

2025

[5] [1892]

Patrick Foley, Matthew Wiesner, Bismarck Odoom, Leibny Paola Garcia Perera, Kenton Murray, and Philipp Koehn

ISCA. Patrick Foley, Matthew Wiesner, Bismarck Odoom, Leibny Paola Garcia Perera, Kenton Murray, and Philipp Koehn. 2024. Where are you from? ge- olocating speech and applications to language iden- tification. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (V...

2024

[6] [2016]

InProceedings of COLING 2016, the 26th International Confer- ence on Computational Linguistics: Technical Pa- pers, pages 3475–3484

Panphon: A resource for mapping IPA seg- ments to articulatory feature vectors. InProceedings of COLING 2016, the 26th International Confer- ence on Computational Linguistics: Technical Pa- pers, pages 3475–3484. ACL. David R Mortensen, Jordan Picone, Xinjian Li, and Kathleen Siminyu. 2021. Tusom2021: A phonet- ically transcribed speech dataset from an en...

2016

[7] [2017]

InInternational conference on machine learning, pages 3319–3328

Axiomatic attribution for deep networks. InInternational conference on machine learning, pages 3319–3328. PMLR. Ashima Suvarna, Harshita Khandelwal, and Nanyun Peng. 2024. Phonologybench: Evaluating phono- logical skills of large language models. InProceed- ings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 1–14. Chihi...

2024

[8] [2018]

Investigating the role of l1 in automatic pro- nunciation evaluation of l2 speech.arXiv preprint arXiv:1807.01738. Joseph Turian, Jordie Shier, Humair Raj Khan, Bhik- sha Raj, Björn W Schuller, Christian J Stein- metz, Colin Malloy, George Tzanetakis, Gissel Ve- larde, Kirk McNally, and 1 others. 2022. Hear: Holistic evaluation of audio representations. I...

Pith/arXiv arXiv 2022

[9] [2020]

InNeuRIPS Workshop on Self- Supervised Learning for Speech and Audio Process- ing

The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken lan- guage modeling. InNeuRIPS Workshop on Self- Supervised Learning for Speech and Audio Process- ing. Ludger Paschen, François Delafontaine, Christoph Draxler, Susanne Fuchs, Matthew Stave, and Frank Seifart. 2020. Building a time-aligned cross- linguistic reference c...

arXiv 2021

[10] [2021]

InIEEE International Conference on Acoustics, Speech, and Signal Processing

The accented english speech recognition chal- lenge 2020: Open datasets, tracks, baselines, results and methods. InIEEE International Conference on Acoustics, Speech, and Signal Processing. Ryan Soh-Eun Shim, Kalvin Chang, and David R Mortensen. 2024. Phonotactic complexity across dialects. InProceedings of the 2024 Joint In- ternational Conference on Com...

2020

[11] [2022]

Please call Stella

Simple and effective zero-shot cross-lingual phoneme recognition. InInterspeech 2022, pages 2113–2117. 12 Wei Xue, Roeland van Hout, Catia Cucchiarini, and Helmer Strik. 2023. Assessing speech intelligibility of pathological speech in sentences and word lists: The contribution of phoneme-level measures.Jour- nal of Communication Disorders, 102:106301. Chi...

Pith/arXiv arXiv 2022

[12] [2023]

International Phonetic Association

Assessment of l2 intelligibility: Comparing l1 listeners and automatic speech recognition.Re- CALL: the Journal of EUROCALL, 35(1):89–104. International Phonetic Association. 1999.Handbook of the International Phonetic Association: A guide to the use of the International Phonetic Alphabet. Cambridge University Press. Heejin Kim, Mark Hasegawa-Johnson, Adr...

Pith/arXiv arXiv 1999