Comparison of End-to-end Speech Assessment Models for the NOCASA 2025 Challenge

Aleksei \v{Z}avoronkov; Tanel Alum\"ae

REVIEW 3 major objections 6 minor 23 references

Comparison of End-to-end Speech Assessment Models for the NOCASA 2025 Challenge

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An end-to-end trainable GOP-CTC model achieved the top leaderboard scores in the NOCASA 2025 pronunciation assessment challenge, with 44.8% UAR and 0.505 MAE on the official test set.

desk verdict A solid, incremental workshop paper with a real empirical result (top NOCASA leaderboard scores) but a significant data-description inconsistency that must be fixed before the leaderboard claim is credible. read the letter →

arxiv 2509.03256 v1 pith:L77LUSDF submitted 2025-09-03 cs.CL cs.SDeess.AS

Aleksei Žavoronkov , Tanel Alumäe This is my paper · ORCID

classification cs.CLcs.SDeess.AS

keywords pronunciationassessmentNOCASA2025goodness-of-pronunciationCTCend-to-endNorwegianL2ordinallossspeechscoring

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper compares three end-to-end systems for word-level pronunciation assessment of children learning Norwegian as a second language in the NOCASA 2025 challenge. Its central finding is that a model which converts alignment-free CTC-based goodness-of-pronunciation features into trainable end-to-end inputs, called GOP-CTC-AF-E2E, outperforms both the other architectures and the official baselines. On the challenge test set, the model reaches 44.8% unweighted average recall (UAR) and 0.505 mean absolute error (MAE), versus baseline scores of 36.4% UAR and 0.55 MAE. The results matter because reliable automatic scoring of L2 child speech could enable feedback at scale without expert human listeners.

What carries the argument

The central object is the set of alignment-free CTC GOP features for each canonical letter $c_i$: the log posterior of the whole canonical sequence (LPP), the vector of log posterior ratios for every possible substitution $\mathrm{LPR}_{sub}(i,j)$, and the scalar log posterior ratio for deleting $c_i$, all computed from CTC emissions and marginalized with the forward algorithm. These values, concatenated with a learned embedding of the letter, form a token sequence fed into a single Transformer layer (the 'GOP Transformer'), max-pooled, and classified into scores 1 to 5. The other load-bearing ingredient is the weighted ordinal cross-entropy loss $\mathcal{L}_{ordinal}(P,y)=\sum_i w_y[-\log(1-p_i)\cdot d(y,i)^\alpha]$, which penalizes errors by their ordinal distance and balances classes; with $\alpha=0.5$ it improves UAR and MAE.

What would settle it

Run the same training pipeline using genuine speaker identities for the train/dev split and compare the selected models' UAR and MAE on the official test set; if the pseudo-speaker clustering had leaked speakers between train and dev, the primary result (44.8% UAR) would not replicate or the dev-based ranking would change.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that pronunciation quality can be scored from substitution and deletion likelihood ratios computed with CTC, without any forced alignment, and that the whole pipeline—feature extraction, letter embeddings, a small Transformer, and the classifier, together with the pretrained CTC backbone—can be fine-tuned jointly for the scoring task. The authors show that freezing the backbone collapses performance on the internal development set (UAR drops from 44.0% to 32.6%), while removing the Transformer or the letter embeddings costs only a few points, and swapping the weighted ordinal cross-entropy for plain cross-entropy also hurts UAR and MAE. The resulting system achieved the top leaderboard scores among all participants.

Load-bearing premise

The load-bearing premise is that the internal development set, created by spectral clustering of speaker embeddings into pseudo-speakers, accurately separates true speakers, since all model selection—checkpoints, class weights, loss scale $\alpha$, and interpolation weights—is tuned against this set.

Editorial extensions

If this is right

Making the CTC backbone trainable is essential: freezing it lowers UAR from 44.0% to 32.6% on the internal development set.
Weighted ordinal cross-entropy with $\alpha=0.5$ gives the best UAR/MAE trade-off, while larger $\alpha$ suppresses predictions of extreme scores 1 and 5.
Interpolating the three models improves development UAR (50.4%) but not test UAR (42.0% vs 44.8%), indicating that dev-based interpolation weights overfit.
Alignment-free GOP features remove the need for external forced-alignment tools, making the method applicable to languages and datasets without phoneme alignments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the same recipe transfers, alignment-free CTC GOP features could be used for phoneme-level or utterance-level feedback and for other L2 languages with pretrained CTC models.
The dev-set split relies on pseudo-speaker clustering; without verification that clusters correspond to speakers, the reported dev-based model rankings may be optimistic, and official test scores could shift if selection were repeated with true speaker labels.
A testable extension is to run the same challenge pipeline using ground-truth speaker information when it is released, to quantify the effect of pseudo-labeling on checkpoint and hyperparameter selection.
The efficiency claim—CTC emissions computed once, then many marginalized likelihoods—suggests the approach can scale to real-time feedback apps, but the paper does not measure latency directly.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The main thing you should know: this paper reports a clear win on the NOCASA 2025 challenge — their GOP-CTC-AF-E2E model beats the official baselines by a wide margin (44.8% UAR vs 36.4% baseline, MAE 0.505 vs 0.55). The core novelty is making the alignment-free CTC GOP features end-to-end trainable, with letter embeddings and a transformer layer on top. The ablations show that end-to-end fine-tuning matters a lot (frozen CTC drops UAR from 44.0 to 32.6), and the weighted ordinal CE loss is a simple, reusable idea. The paper is well written and the negative result on dev-based interpolation (submission #5 overfitting) is honestly reported.

Now the soft spots. The stress-test note is correct: the test-set size is internally inconsistent. The introduction says 1,930 test utterances from 8 speakers; Section 3.1 says 1,460. That’s a 470-utterance gap, not a typo-level error. Since the headline claim is about beating baselines on the official test set, the reader genuinely cannot tell which number is right or whether the evaluation was done on the same set described. This needs an explicit correction or the experimental basis is ambiguous. Second, there are no confidence intervals or significance tests. On a test set from only 8 speakers, a 8.4-point UAR gap is likely real, but with that few speakers the variance could be large. Third, the pseudo-speaker clustering for the dev split is unvalidated — speaker leakage between train and dev could bias checkpoint selection and hyperparameters, though the final leaderboard numbers come from the official server so this is a secondary issue. Finally, no code or data release, which limits reproducibility.

The citations look reasonable: they build on Cao et al.'s CTC GOP work and cite the challenge description, and the self-citation to Alumäe's MADASR system is legitimately relevant for the prefix-tuning variant.

This is a workshop-style paper (IEEE MLSP) and it does what a good challenge paper should: clear architecture comparison, honest ablations, and a strong official result. The central idea is likely correct, but the data inconsistency and missing error bars need addressing. I'd send it to peer review, but with the expectation that the authors clean up the data description and add uncertainty estimates before publication.

Reading group: worth a look if anyone in your group does speech assessment or uses ordinal losses. Otherwise, it's a fine skim.

Referee Report

3 major / 6 minor

Summary. This paper reports the TalTech submission to the NOCASA 2025 pronunciation assessment challenge. It describes three end-to-end models: a Siamese encoder-decoder model (E2E-R), a prefix-tuned wav2vec2.0 classification model, and a novel GOP-CTC-AF-E2E model that makes alignment-free CTC-based goodness-of-pronunciation features end-to-end trainable. The paper also proposes a weighted ordinal cross-entropy loss aimed at optimizing UAR and MAE. The authors report development-set comparisons, an ablation study, and five official leaderboard submissions. Their primary system (submission #4) is claimed to achieve top leaderboard scores with UAR 44.8% and MAE 0.505, surpassing the challenge baselines (UAR 36.4%, MAE 0.55).

Significance. The strongest asset of the paper is that the headline result comes from the official NOCASA leaderboard rather than from a self-constructed test set, which is a meaningful form of external validation. The ablation showing that freezing the pretrained CTC model drops UAR from 44.0% to 32.6% is a clear, falsifiable demonstration that end-to-end fine-tuning is important. The idea of making alignment-free GOP features differentiable and trainable is a useful methodological contribution. I do not see a circularity issue: the GOP features are computed from a pretrained model, after which all parameters are fine-tuned on human scores; no already-fitted quantity is reused to certify the result. However, the manuscript is not currently reproducible as a data description because of the internal inconsistency in training/test sizes and the unvalidated pseudo-speaker splitting procedure.

major comments (3)

[Section 1 vs Section 3.1] The paper gives incompatible dataset sizes in two places. Section 1 states a training corpus of 10,334 utterances and a test set of 1,930 utterances from 8 speakers, while Section 3.1 states that the provided dataset consists of 7,857 labeled training utterances and 1,460 unlabeled test utterances. Since Table 2 reports official leaderboard scores, the reader cannot tell whether the leaderboard was evaluated on the same test set described in Section 3.1. If the official test set is 1,930 utterances, then Section 3.1 describes a different, smaller evaluation set and the leaderboard comparison is not reproducible from the manuscript alone. If the official test set is 1,460 utterances, the introduction is wrong by 470 utterances. Please reconcile these numbers and explicitly state which split was used for each submission.
[Section 3.1] All model selection decisions (checkpoint selection by dev UAR, interpolation weights, loss alpha, class weights) are made on an internal development set obtained by unsupervised speaker clustering into pseudo-speakers. The paper provides no verification that the pseudo-speaker clusters correspond to true speakers. If a true speaker appears in both the training and development clusters, the development-set metrics are optimistic and the selected hyperparameters may not transfer to the official test set. The performance drop of submission #5 from 50.4% UAR on the development set to 42.0% on the test set is consistent with development overfitting. Please report cluster-quality diagnostics where possible, or at minimum show sensitivity of the main conclusions to alternative split seeds and clustering parameters.
[Section 2.2, Eq. (1)] The weighted ordinal cross-entropy loss as written is under-specified. The summation runs over all classes i, including i = y, where d(y,y)=0. The term for i=y is therefore 0 times -log(1-p_y), which is undefined in the limit p_y -> 1. If the intended loss is a distance-weighted complement loss over i != y, the equation should state that explicitly; if conventional ordinal cross-entropy was intended, the formula should be corrected. As written, the method section does not support the claim that the loss is a cross-entropy-based objective, and a reader cannot reproduce the training loss from the formula.

minor comments (6)

[Section 3.3 and Section 3.4] The table references are inconsistent: Section 3.3 says 'Table 3 lists results' for the development-set results, which is actually Table 1, and the submission table is referred to as 'Table 3.2' rather than Table 2. Section 3.4 refers to 'Table 3.4' for the ablation table, which should be Table 3.
[Section 3.3] There is a typo in the numbered list: 'Predictons' should be 'Predictions'.
[Section 2.2] The text says 'mean average error (MAE)', but the metric is described elsewhere as mean absolute error; please use the correct term consistently.
[Section 3.1] The test set is called 'unlabeled' in Section 3.1, but Section 1 says word transcripts are provided for the test set. Please clarify that only pronunciation scores are withheld.
[Section 1] The abstract and introduction state that the system achieved top leaderboard scores, but Table 2 only lists the authors' own submissions and challenge baselines. Please include the final leaderboard rank or a link to the official leaderboard so this claim is verifiable.
[Section 3.3] The paper reports single evaluation numbers without confidence intervals or significance tests. Given the small number of official submissions allowed, a bootstrap over the development set or a statement about evaluation variance would help the reader assess whether the differences between submissions are meaningful.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the GOP-CTC-AF-E2E derivation is self-contained, and the only self-citation is non-load-bearing.

full rationale

The paper's central claim is an empirical leaderboard result, not a derived quantity. The GOP-CTC-AF-E2E model computes alignment-free CTC features from a pretrained model (Section 2.1.3) and then fine-tunes all components end-to-end against human pronunciation scores; this is supervised training, not a prediction that reduces to its inputs. The weighted ordinal CE loss (Section 2.2) is a defined training objective with independent UAR/MAE evaluation on the dev and test sets, so there is no fitted-parameter-renamed-as-prediction pattern. The only self-citation, [6] (Alumäe et al., MADASR), is used merely to note a related prefix-vector application; the actual prefix-tuning method is attributed to Li and Liang [5], and the paper explicitly differs from [6] by training prefixes jointly with the model. The GOP feature idea is cited to external authors [2], and the claimed novelty (end-to-end trainability) is supported by the ablation showing a large drop when the CTC model is frozen (32.6% vs 44.0% UAR), so the central contribution has independent empirical content. The manuscript contains a data-description inconsistency (1,930 test utterances in the introduction vs 1,460 in Section 3.1), which is a reproducibility/correctness concern rather than a circularity, and does not affect this score.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The model pipeline rests on a handful of standard domain assumptions about pretrained models and human labels, plus one ad hoc assumption about pseudo-speaker clustering. No physically new entities are introduced.

free parameters (3)

Loss distance exponent α = 0.5 (and 1.5 for submission 1)
Set by hand; found that larger α avoided extreme predictions.
Interpolation weights for model ensemble = Optimized on internal dev set (e.g., 0.1/0.9 for submissions 3 and 5)
Weights for combining model outputs were chosen to maximize dev UAR.
Speaker clustering hyperparameters = p=0.01 pruning, K range 40-45, 20% dev speakers
These are chosen heuristically to create pseudo-speaker clusters for the train/dev split.

assumptions (5)

domain assumption The NbAiLab pretrained wav2vec2 models provide transferable representations for Norwegian child speech.
Used as the backbone for all models; no adaptation to child speech is demonstrated.
ad hoc to paper Speaker recognition embeddings cluster utterances by true speaker identity.
The train/dev split is built from these clusters; if clustering is imperfect, dev results are inflated.
domain assumption The CTC letter vocabulary matches the orthographic transcriptions of the target words.
GOP feature computation depends on forming substitution/deletion sequences from canonical letter labels.
domain assumption Human expert scores (1-5) are treated as ground truth, and class frequencies in the training set are a fair basis for weighting.
The weighted ordinal CE loss uses these labels and inverse class frequencies.
domain assumption The official challenge test set is a representative sample of the target population.
Used as the evaluation set, but only 8 speakers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Comparison of End-to-end Speech Assessment Models for the NOCASA 2025 Challenge." pith.science (2026). https://pith.science/paper/L77LUSDF

@misc{pith2026250903256,
  author       = {Pith},
  title        = {Pith review of: Comparison of End-to-end Speech Assessment Models for the NOCASA 2025 Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L77LUSDF}},
  note         = {Machine review of arXiv:2509.03256}
}

read the original abstract

This paper presents an analysis of three end-to-end models developed for the NOCASA 2025 Challenge, aimed at automatic word-level pronunciation assessment for children learning Norwegian as a second language. Our models include an encoder-decoder Siamese architecture (E2E-R), a prefix-tuned direct classification model leveraging pretrained wav2vec2.0 representations, and a novel model integrating alignment-free goodness-of-pronunciation (GOP) features computed via CTC. We introduce a weighted ordinal cross-entropy loss tailored for optimizing metrics such as unweighted average recall and mean absolute error. Among the explored methods, our GOP-CTC-based model achieved the highest performance, substantially surpassing challenge baselines and attaining top leaderboard scores.

Figures

Figures reproduced from arXiv: 2509.03256 by the authors.

**Figure 3.** Architecture of the end-to-end model using CTC-based alignment-free goodness-of-pronunciation features (GOP-CTC-AF-E2E). θp. Following [5], the prefix vectors are prepended to each Transformer layer as additional key and value vectors. These vectors do not use positional encoding, and their corresponding outputs are discarded after the Transformer layer. The actual letter sequence of a word is not used in this mod… view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 22 canonical work pages

[1]

Such systems can be used to provide feedback in computer-aided language learning applications

INTRODUCTION The task of speech pronunciation assessment focuses on au- tomatically evaluating a language learner’s pronunciation of phonemes, words, or complete utterances. Such systems can be used to provide feedback in computer-aided language learning applications. The Non-native Children’s Automatic Speech Assessment (NOCASA) Challenge [1] was designe...

work page
[2]

Comparison of End-to-end Speech Assessment Models for the NOCASA 2025 Challenge

METHODS 2.1. Models 2.1.1. Encoder-decoder Siamese model (E2E-R) We adapted the E2E-R model proposed in [3], which fol- lows a two-stage training strategy (see Figure 1). In the first stage, a pre-trained self-supervised learning (SSL) model is fine-tuned for phoneme recognition using a hybrid encoder- decoder CTC-attention mechanism. The second stage int...

work page Pith review arXiv 2025
[3]

GOP Transformer

For example, when using the large XLS-R 1B wav2vec2.0 model with a hidden size of 1280 and 48 Transformer layers, and setting the prefix length to 2, this results in approxi- mately 122K additional parameters per word type. A related method was used in [6] to incorporate dialect information into a wav2vec2.0-based speech recognition model. Unlike that wor...

work page
[4]

eigen- gap

EXPERIMENTAL RESULTS 3.1. Data The dataset provided by the challenge organizers consists of 7857 labeled training utterances and 1460 unlabeled test ut- terances. Due to privacy constraints, speaker identities are not disclosed. To enable model tuning, we split the training data into internal training and development sets. To prevent overfitting, it is cr...

work page
[5]

GOP-CTC-AF-E2E model, trained on the internal training split using ordinal CE loss with distance penalty scalerα = 1.5, without class weights

work page
[6]

Same as #1, but trained on the full official training set

work page
[7]

Predictons of #2 interpolated with a E2E-R model, with interpolation weights (0.1, 0.9) optimized on the inter- nal development set to maximize UAR

work page
[8]

GOP-CTC-AF-E2E model, trained using weighted or- dinal CE loss (α = 0.5) on the full training set

work page

Show all 23 references

[9]

Since submission #4 resulted in the best UAR and MAE scores, we used this as our primary result in the leaderboard

Interpolation of all three models (listed in Table 3), weights optimized on development data. Since submission #4 resulted in the best UAR and MAE scores, we used this as our primary result in the leaderboard. On the internal development set, the three-model interpola- tion (#...
[10]

CONCLUSION We investigated three distinct end-to-end approaches for au- tomatic pronunciation assessment within the NOCASA 2025 Challenge framework. Our best-performing GOP-CTC-AF- E2E model successfully integrates alignment-free CTC-based GOP features with Transformer-based c...

2025
[11]

Non-native children’s automatic speech assessment challenge (NOCASA),

Yaroslav Getman, Tam´as Gr´osz, Mikko Kurimo, and Gi- ampiero Salvi, “Non-native children’s automatic speech assessment challenge (NOCASA),” arXiv preprint arXiv:2504.20678, 2025

2025 arXiv
[12]

A framework for phoneme-level pro- nunciation assessment using CTC,

Xinwei Cao, Zijian Fan, Torbjørn Svendsen, and Gi- ampiero Salvi, “A framework for phoneme-level pro- nunciation assessment using CTC,” in Proc. Inter- speech, 2024, pp. 302–306

2024
[13]

Fine-tuning self-supervised learning models for end-to-end pronunciation scoring,

Ahmed I. Zahran, Aly A. Fahmy, Khaled T. Wassif, and Hanaa Bayomi, “Fine-tuning self-supervised learning models for end-to-end pronunciation scoring,”IEEE Ac- cess, vol. 11, pp. 112650–112663, 2023

2023
[14]

NB Tale - speech database for Norwegian,

Lingit AS and National Library of Norway (Spr˚akbanken), “NB Tale - speech database for Norwegian,” https://hdl.handle.net/21. 11146/31, 2013

2013
[15]

Prefix-tuning: Op- timizing continuous prompts for generation,

Xiang Lisa Li and Percy Liang, “Prefix-tuning: Op- timizing continuous prompts for generation,” in Proc. ACL-IJCNLP (Volume 1: Long Papers) , 2021, pp. 4582–4597

2021
[16]

Dialect adaptation and data augmentation for low- resource ASR: TalTech systems for the MADASR 2023 challenge,

Tanel Alum ¨ae, Jiaming Kong, and Daniil Robnikov, “Dialect adaptation and data augmentation for low- resource ASR: TalTech systems for the MADASR 2023 challenge,” in Proc. ASRU. IEEE, 2023, pp. 1–7

2023
[17]

Wespeaker: A research and production oriented speaker embedding learning toolkit,

Hongji Wang, Chengdong Liang, Shuai Wang, Zhengyang Chen, Binbin Zhang, Xu Xiang, Yanlei Deng, and Yanmin Qian, “Wespeaker: A research and production oriented speaker embedding learning toolkit,” in Proc. ICASSP. IEEE, 2023, pp. 1–5

2023
[18]

Advancing speaker embedding learning: Wespeaker toolkit for research and production,

Shuai Wang, Zhengyang Chen, Bing Han, Hongji Wang, Chengdong Liang, Binbin Zhang, Xu Xiang, Wen Ding, Johan Rohdin, Anna Silnova, et al., “Advancing speaker embedding learning: Wespeaker toolkit for research and production,” Speech Communication , vol. 162, pp. 103104, 2024

2024
[19]

Simple attention module based speaker verification with iterative noisy label detection,

Xiaoyi Qin, Na Li, Chao Weng, Dan Su, and Ming Li, “Simple attention module based speaker verification with iterative noisy label detection,” in Proc. ICASSP. IEEE, 2022, pp. 6722–6726

2022
[20]

V oxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,

Yuke Lin, Ming Cheng, Fulin Zhang, Yingying Gao, Shilei Zhang, and Ming Li, “V oxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,” arXiv preprint arXiv:2407.11510, 2024

2024 arXiv
[21]

V oxCeleb2: Deep speaker recognition,

Joon Son Chung, Arsha Nagrani, and Andrew Zisser- man, “V oxCeleb2: Deep speaker recognition,” Proc. Interspeech, 2018

2018
[22]

Silero V AD: pre-trained enterprise- grade voice activity detector (V AD), number detector and language classifier,

Silero Team, “Silero V AD: pre-trained enterprise- grade voice activity detector (V AD), number detector and language classifier,” https://github.com/ snakers4/silero-vad, 2024

2024
[23]

Collecting linguistic resources for assessing children‘s pronunci- ation of Nordic languages,

Anne Marte Haug Olstad, Anna Smolander, Sofia Str¨ombergsson, Sari Ylinen, Minna Lehtonen, Mikko Kurimo, Yaroslav Getman, Tam´as Gr ´osz, Xinwei Cao, Torbjørn Svendsen, and Giampiero Salvi, “Collecting linguistic resources for assessing children‘s pronunci- ation of Nordic lan...

2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Such systems can be used to provide feedback in computer-aided language learning applications

INTRODUCTION The task of speech pronunciation assessment focuses on au- tomatically evaluating a language learner’s pronunciation of phonemes, words, or complete utterances. Such systems can be used to provide feedback in computer-aided language learning applications. The Non-native Children’s Automatic Speech Assessment (NOCASA) Challenge [1] was designe...

work page

[2] [2]

Comparison of End-to-end Speech Assessment Models for the NOCASA 2025 Challenge

METHODS 2.1. Models 2.1.1. Encoder-decoder Siamese model (E2E-R) We adapted the E2E-R model proposed in [3], which fol- lows a two-stage training strategy (see Figure 1). In the first stage, a pre-trained self-supervised learning (SSL) model is fine-tuned for phoneme recognition using a hybrid encoder- decoder CTC-attention mechanism. The second stage int...

work page Pith review arXiv 2025

[3] [3]

GOP Transformer

For example, when using the large XLS-R 1B wav2vec2.0 model with a hidden size of 1280 and 48 Transformer layers, and setting the prefix length to 2, this results in approxi- mately 122K additional parameters per word type. A related method was used in [6] to incorporate dialect information into a wav2vec2.0-based speech recognition model. Unlike that wor...

work page

[4] [4]

eigen- gap

EXPERIMENTAL RESULTS 3.1. Data The dataset provided by the challenge organizers consists of 7857 labeled training utterances and 1460 unlabeled test ut- terances. Due to privacy constraints, speaker identities are not disclosed. To enable model tuning, we split the training data into internal training and development sets. To prevent overfitting, it is cr...

work page

[5] [5]

GOP-CTC-AF-E2E model, trained on the internal training split using ordinal CE loss with distance penalty scalerα = 1.5, without class weights

work page

[6] [6]

Same as #1, but trained on the full official training set

work page

[7] [7]

Predictons of #2 interpolated with a E2E-R model, with interpolation weights (0.1, 0.9) optimized on the inter- nal development set to maximize UAR

work page

[8] [8]

GOP-CTC-AF-E2E model, trained using weighted or- dinal CE loss (α = 0.5) on the full training set

work page