Pith. sign in

REVIEW 4 major objections 4 minor 49 references

Cross-modal Knowledge Transfer Learning as Graph Matching Based on Optimal Transport for ASR

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Modeling cross-modal alignment as graph matching with fused Wasserstein and Gromov-Wasserstein costs lowers AISHELL-1 test CER to 3.98% from 5.76% (Conformer+CTC) and 4.19% (prior OT), with no external LM at inference.

desk verdict A plausible but unproven extension of the authors' own OT-based ASR transfer work: the claimed gains from edge-level graph matching are not isolated by any ablation, so the central novelty remains untested. read the letter →

arxiv 2505.13079 v1 pith:OWHV4ZD4 submitted 2025-05-19 eess.AS cs.AI

Xugang Lu , Peng Shen , Yu Tsao , Hisashi Kawai This is my paper · ORCID
classification eess.AScs.AI
keywords optimaltransportGromov-WassersteindistancegraphmatchingknowledgetransferCTCASRpretrainedlanguagemodelcross-modalalignmentAISHELL-1
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that prior optimal-transport-based linguistic knowledge transfer for ASR is limited because it treats speech and text features as unordered sets. It proposes Graph Matching Optimal Transport (GM-OT), which represents each utterance as a graph in acoustic and linguistic space, aligns nodes with Wasserstein distance, aligns edges with Gromov-Wasserstein distance, and folds both into one fused cost with a temporal-consistency penalty. On AISHELL-1 Mandarin ASR, the best GM-OT setting reaches 3.98% test CER, against 5.76% for Conformer+CTC and 4.19% for the $\alpha=0$ special case approximating prior OT transfer. The authors argue that existing OT-based knowledge transfer is a special case of this framework, and that structural and temporal terms, not the PLM alone, drive the improvement. If true, a CTC encoder can absorb linguistic knowledge during training and still decode at plain CTC speed without an external language model.

What carries the argument

The central object is the fused Gromov-Wasserstein distance (FGWD), defined as a weighted sum of a node-matching Wasserstein term and an edge-matching Gromov-Wasserstein term: $\mathcal{L}_{\mathrm{FGWD}} = \min_{\gamma} (1-\alpha)\langle D_{AL},\gamma\rangle + \alpha \langle D(D_A,D_L)\otimes\gamma,\gamma\rangle$, with a temporal-consistency cost $\rho |i/l_a - j/l_t|^2$ folded into the cross-modal ground cost. It carries the argument because it turns the transport coupling from a flat node-to-node match into a coupling that also respects intra-sequence distances, so adjacent acoustic frames stay near neighboring linguistic tokens. The optimal coupling is then used to project acoustic features into the linguistic space, where a cosine alignment loss and the fused distance train the acoustic encoder.

What would settle it

An independent rerun of Table 1 on both the original and the corrected AISHELL-1 transcripts, comparing Setting 5 against Setting 1 ($\alpha=0$) and Conformer+CTC under identical scoring, would settle the claim; if the 3.98% vs 4.19% gap disappears or reverses when baselines are scored on corrected text, the central claim is unsupported. A second check is to vary $\alpha$ upward from 0 with $\rho=\beta=0$: if CER never improves over $\alpha=0$, the Gromov-Wasserstein edge term is not the active ingredient.

Watch

Extended reading notes

Core claim

Stated on the paper's own terms: aligning acoustic and linguistic sequences as ordered graphs transfers pretrained-language-model knowledge into the acoustic encoder more efficiently than aligning them as unordered sets. The paper defines the alignment cost as a fused Gromov-Wasserstein distance, a weighted combination of a node-level Wasserstein cost and an edge-level Gromov-Wasserstein cost over the two graphs' cosine-distance matrices, plus a diagonal temporal-consistency penalty. With this objective and a Sinkhorn solver, the best configuration (Setting 5) yields 3.98% CER on the AISHELL-1 test set, down from 5.76% for the Conformer+CTC baseline and 4.19% for the $\alpha=0$ special case. The authors read this as evidence that edge structure and temporal monotonicity, not just feature geometry, are what make cross-modal knowledge transfer effective for CTC-based ASR.

Load-bearing premise

The load-bearing assumption is that the authors' unreleased corrections to the AISHELL-1 reference transcripts are correct and that published baselines, evaluated on the original references, would not gain equally from the same corrections; otherwise the reported CER gaps may be label-change artifacts.

Editorial extensions

If this is right

  • A CTC-based recognizer can absorb pretrained-language-model knowledge at training time only; at inference the acoustic branch alone runs, so decoding speed stays comparable to CTC greedy search with no external LM.
  • Because Setting 1 ($\alpha=0$) reproduces prior OT-based transfer, the improved CER of Settings 2-5 isolates the contribution of edge matching and temporal consistency rather than merely adding a PLM or adapter.
  • The estimated transport coupling provides token-level alignments between acoustic frames and linguistic tokens, and Figure 3 shows these alignments sharpen or spread as $\alpha$, $\rho$, and $\beta$ vary.
  • Performance is sensitive to the joint choice of $\alpha$, $\rho$, $\beta$, and $w_s$; the best result occurs at small $\alpha$ (0.02) with moderate temporal and entropy regularization, while larger $\alpha$ degrades CER.
  • The framework subsumes the prior OT method as the $\alpha=0$ special case, so one training recipe covers a family of alignment objectives from node-only to edge-only matching.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The graph-matching formulation is not specific to CTC; it could be inserted as an auxiliary loss in any encoder that must align two ordered modalities, such as speech-to-text translation or audiovisual speech recognition.
  • A direct test of the mechanism would be to replace the learned coupling with a purely diagonal alignment while keeping the Gromov-Wasserstein loss; if CER does not revert toward the Setting 1 level, the benefit may come from the temporal penalty rather than from edge matching itself.
  • The paper's own sensitivity analysis suggests $\alpha$, $\rho$, and $\beta$ interact, so an automatic schedule or efficient search over these three is the natural next practical step, especially since the conclusion flags hyperparameter sensitivity as the main difficulty.
  • Since the transport coupling assigns acoustic frames to linguistic tokens, the same GM-OT objective could double as a forced-alignment or duration-estimation module, an application the paper does not develop.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes GM-OT, a graph-matching optimal transport framework for transferring linguistic knowledge from a pretrained language model (BERT) into a CTC-based acoustic encoder for Mandarin ASR. The method models the acoustic and linguistic sequences as two complete graphs whose nodes are frame/token embeddings and whose edges are cosine-distance similarities; node alignment is encouraged through a Wasserstein distance, edge alignment through a Gromov-Wasserstein distance, and the two are fused with a tunable weight alpha. A temporal-consistency cost is added to the node cost matrix, and the final training objective combines the CTC loss with the fused alignment loss. Experiments on AISHELL-1 report a test CER of 3.98% for the best configuration, down from 5.76% for a Conformer+CTC baseline and 4.19% for the alpha=0 special case. The paper also claims that prior OT-based linguistic knowledge transfer can be derived as a special case (alpha=0) of the proposed framework.

Significance. If the empirical claims were robust, the paper would offer a useful structural extension of OT-based knowledge transfer for E2E-ASR, with the distinctive property of aligning both nodes and edges of the two modalities. The special-case reduction alpha=0 in Eq. (12) is transparent and correct, and the paper honestly acknowledges its own prior OT work [28] as the special case, which is a point in its favor. However, the magnitude of the reported improvement is small (0.21% absolute CER on test over the special case), no ablation isolates the edge-matching GWD term, and the paper's reliance on unreleased corrected reference transcriptions weakens the reproducibility of the headline numbers. The central contribution is therefore plausible but not yet demonstrated.

major comments (4)
  1. [Section 3.4, Table 1 and Table 2] The claim that 'GM-OT efficiently transfers linguistic knowledge ... yielding significant performance improvements' is not supported by the experiments as presented, because no configuration isolates the Gromov-Wasserstein edge-matching term. The best result (Setting 5, 3.98% test CER) differs from the alpha=0 setting (Setting 1, 4.19%) in three hyperparameters at once: alpha changes from 0 to 0.02, rho from 0 to 0.3, and beta from 0.05 to 0.5 (Table 2). Thus the 0.21% absolute CER gain may be entirely due to the temporal-consistency cost rho or the entropy-regularization change beta, rather than to the GWD term. Please provide at least two additional runs: one with alpha>0 while rho=0 (and beta, w_s matched to Setting 5), and one with rho>0 while alpha=0 (and beta, w_s matched to Setting 5). In addition, no error bars, multiple seeds, or significance tests are reported; a 0.2% CER difference across a 130-epoch training run may be within run-to-run variance.
  2. [Section 3.4, reference-correction statement] The paper states that the authors 'checked and corrected the reference transcriptions' for the development and test sets and that all reported results are calculated on the corrected references, but the corrections are not released. Without the corrected references (or a diff), an independent group cannot reproduce the reported CER values or verify that the same corrections were applied equally to every comparison system in Table 1. Please release the corrected reference files and explicitly confirm that all baselines in Table 1 (including the light-gray rows) were evaluated on exactly the same corrected references. If the published baselines from the literature were evaluated on the original references, the comparison against them is not apples-to-apples.
  3. [Eq. (15), Section 2.3] Equation (15) defines the projected linguistic feature as \tilde{Z}_L = \gamma^* \times H_A \in \mathbb{R}^{l_t \times d_t}, but the dimensions do not conform: \gamma^* \in \mathbb{R}^{l_a \times l_t} and H_A \in \mathbb{R}^{l_a \times d_t} cannot be multiplied in this order. The intended expression is presumably (\gamma^*)^T H_A. As written, the central formula for the alignment loss L_align is technically undefined; please correct it and verify that the subsequent loss computation in Eq. (16) uses the transposed coupling.
  4. [Section 2.2.3, Eqs. (13)-(14)] The iterative rule for approximating the Fused Gromov-Wasserstein distance is asserted without a convergence proof or a matching citation. Equation (13) resembles a proximal-point update, but Eq. (14) then updates the cost matrix using the latest coupling, and the paper does not state whether this alternating scheme converges to the solution of Eq. (12). Please provide a formal convergence statement under appropriate assumptions, or explicitly state that the solver is a heuristic and report its sensitivity to the number of iterations and to beta. Without this, the 'LFGWD' minimizer used in Eq. (18) is not well defined.
minor comments (4)
  1. [Section 3.4] The sentence 'Setting 1 is similar as in others work [28]' is grammatically incorrect and imprecise; specify exactly which components of Setting 1 reproduce [28] (e.g., alpha=0 removes the GWD term but the temporal cost rho is also 0).
  2. [Section 2.3, Eq. (18)] The total loss includes both L_align and L_FGWD, but the paper does not state how the coupling gamma* used in L_align is obtained (the one that minimizes L_FGWD?) nor whether gradients are backpropagated through the Sinkhorn iterations; please clarify the training procedure.
  3. [Throughout the paper] There are many typos and spelling errors (e.g., 'languaue' in the Index Terms, 'T e chnology' in the affiliation, 'pair-wised' in the text, and an obvious year error in reference [38] 'vol. 13, no. 9, 212, 220'). A thorough proofreading is needed.
  4. [Table 1 caption] The text notes that the baseline comparisons are from the authors' own implementations; this should be stated clearly in the table caption, and the hyperparameters used for the baselines should be given.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GM-OT's derivation is self-contained; the alpha=0 special-case claim follows directly from Eq. (12), and self-citations are used only as baselines.

full rationale

The paper's derivation chain does not reduce to its own inputs. The load-bearing alignment losses in Eqs. (4)-(12) are defined from external OT, GWD, and FGWD references [36-38], and the claim that prior OT-based methods are a special case is literally Eq. (12) at alpha=0, a transparent nesting relation rather than a prediction obtained from the input. The ASR experiments are evaluated on an external benchmark, AISHELL-1, and hyper-parameters are empirically chosen and then evaluated on a held-out test set, so no fitted parameter is renamed as a prediction. Self-citations [27-29] appear as baselines and predecessor models, not as forced support for the novelty claim, and no uniqueness theorem is imported. The stated limitation of hyper-parameter sensitivity, the corrected-reference procedure, and the absence of ablations isolating the GWD term are empirical robustness and reproducibility concerns, not circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard optimal transport and Gromov-Wasserstein mathematics, the graph modeling of utterances, a temporal monotonicity assumption, and the authors' corrected AISHELL-1 references. Four hyperparameters (alpha, rho, beta, ws) plus lambda are hand-set or tuned on the same corpus, so the marginal effect of graph matching alone is not separately quantified.

free parameters (5)
  • alpha (fused-distance weight) = 0.02 (best test, Setting 5)
    Weights GWD versus WD in Equation (11); searched over 0, 0.01, 0.02, 0.05, and 0.1 in Table 2.
  • rho (temporal consistency weight) = 0.3 (best test, Setting 5)
    Controls the diagonal temporal penalty added in Section 2.2.4; searched over 0.1, 0.3, and 0.5.
  • beta (entropy regularization) = 0.5 (best test, Setting 5)
    Regularizes the transport couplings in Equations (7) and (13); searched over 0.05, 0.3, and 0.5.
  • ws (transferred-feature scale) = 0.1 (best test, Setting 5)
    Scales the linguistic projection added to acoustic features in Equation (17); searched over 0.05, 0.1, and 0.3.
  • lambda (CTC loss weight) = 0.3
    Balances CTC and cross-modal losses in Equation (18); fixed without sensitivity analysis.
assumptions (4)
  • standard math Optimal transport, entropy-regularized Sinkhorn, Gromov-Wasserstein, and Fused Gromov-Wasserstein formulations are valid and computable as in cited works [36,37,38].
    Used to define WD, GWD, FGWD, and the Sinkhorn iterations in Section 2.2.
  • domain assumption Representing an utterance as two graphs, with nodes as feature embeddings and edges as cosine-distance relations, captures the structure needed for cross-modal alignment.
    Core modeling choice in Section 2.2; no evidence is given that cosine-distance edges are the right graph topology for speech-text alignment.
  • domain assumption Acoustic frames and linguistic tokens are temporally coherent, so large time gaps should be penalized and alignment should stay near the diagonal.
    Introduced in Section 2.2.4; the paper itself notes this can be a double-edged sword because token durations are not uniform.
  • ad hoc to paper The AISHELL-1 development and test reference transcriptions contain errors, and the authors' corrections are the correct ground truth for all systems compared.
    Section 3.4 states all results use corrected references, but the corrections are not released or described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-modal Knowledge Transfer Learning as Graph Matching Based on Optimal Transport for ASR." pith.science (2026). https://pith.science/paper/OWHV4ZD4

@misc{pith2026250513079,
  author       = {Pith},
  title        = {Pith review of: Cross-modal Knowledge Transfer Learning as Graph Matching Based on Optimal Transport for ASR},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OWHV4ZD4}},
  note         = {Machine review of arXiv:2505.13079}
}
read the original abstract

Transferring linguistic knowledge from a pretrained language model (PLM) to acoustic feature learning has proven effective in enhancing end-to-end automatic speech recognition (E2E-ASR). However, aligning representations between linguistic and acoustic modalities remains a challenge due to inherent modality gaps. Optimal transport (OT) has shown promise in mitigating these gaps by minimizing the Wasserstein distance (WD) between linguistic and acoustic feature distributions. However, previous OT-based methods overlook structural relationships, treating feature vectors as unordered sets. To address this, we propose Graph Matching Optimal Transport (GM-OT), which models linguistic and acoustic sequences as structured graphs. Nodes represent feature embeddings, while edges capture temporal and sequential relationships. GM-OT minimizes both WD (between nodes) and Gromov-Wasserstein distance (GWD) (between edges), leading to a fused Gromov-Wasserstein distance (FGWD) formulation. This enables structured alignment and more efficient knowledge transfer compared to existing OT-based approaches. Theoretical analysis further shows that prior OT-based methods in linguistic knowledge transfer can be viewed as a special case within our GM-OT framework. We evaluate GM-OT on Mandarin ASR using a CTC-based E2E-ASR system with a PLM for knowledge transfer. Experimental results demonstrate significant performance gains over state-of-the-art models, validating the effectiveness of our approach.

Figures

Figures reproduced from arXiv: 2505.13079 by the authors.

Figure 1
Figure 1. The proposed cross-modal alignment and knowledge transfer model for ASR based on GM-OT. 2.1. Feature exploration in acoustic and linguistic encoders In acoustic encoder, convolution-based down-sampling is ap￾plied in order to reduce the duration lengths of acoustic se￾quence (a down-sampling rate of 4 was used in this paper). Af￾ter down-sampling, a positional encoding is applied in order to keep temporal position s… view at source ↗
Figure 3
Figure 3. Coupling between acoustic feature and linguistic tokens with changing different parameters, (a) weighting between WD and GWD (varying α), (b) controlling temporal consistency between cross-modalities (varying ρ) (fixed α = 0.1), (c) with entropy regularization (varying β) (fixed α = 0.1, ρ = 0.5). 3.3. Behaviors of the GM-OT based feature alignments Before conducting ASR experiments, we investigate the behav￾iors of… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 40 canonical work pages

  1. [28]

    Improving CTC-Based Speech Recognition Via Knowl- edge Transferring from Pre-Trained Language Models,

    K. Deng, S. Cao, Y . Zhang, L. Ma, G. Cheng, J. Xu, P . Zhang, “Improving CTC-Based Speech Recognition Via Knowl- edge Transferring from Pre-Trained Language Models,” in Proc. of ICASSP, pp. 8517-8521, 2022

    work page 2022

  2. [1]

    National Institute of Information and Communications Te chnology, Japan

    work page

  3. [2]

    However, aligning representations betwee n linguistic and acoustic modalities remains a challenge due to inherent modality gaps

    Research Center for Information Technology Innovation, Academia Sinica, Taiwan xugang.lu@nict.go.jp Abstract Transferring linguistic knowledge from a pretrained lan- guage model (PLM) to acoustic feature learning has proven effective in enhancing end-to-end automatic speech recogn i- tion (E2E-ASR). However, aligning representations betwee n linguistic an...

    work page

  4. [3]

    Introduction The pretrained language models (PLMs) learned from large text corpora encodes rich linguistic knowledge, and has greatly ad- vanced the performance of automatic speech recognition (AS R) with end-to-end (E2E) acoustic models [1, 2, 3, 4, 5]. While e n- hancing ASR in decoding, typically as external language mod - els (LMs) for tasks like beam...

    work page

  5. [4]

    Proposed method The proposed model framework is illustrated in Fig. 1. In thi s framework, two encoders, i.e., acoustic encoder and lingui s- tic encoder are used for exploring acoustic and linguistic f ea- ture representations. Between the two encoders, there is on e ‘Adapter’ module to align acoustic feature dimension with t he linguistic one (gray bloc...

    work page Pith review arXiv 2025

  6. [5]

    The dataset consists of: A training set with 340 speakers (150 hours of speech)

    Experiments and results We conducted ASR experiments to evaluate the proposed al- gorithm on the AISHELL-1 dataset [40], a widely used open- source Mandarin speech corpus. The dataset consists of: A training set with 340 speakers (150 hours of speech). A devel - opment set (validation) with 40 speakers (10 hours of speech ). A test set with 20 speakers (5...

    work page

  7. [6]

    By structuring feature embeddings as nodes and capturing sequential relationships as edges, GM-OT enable s a more comprehensive alignment through graph-based optimal transport

    Conclusion and future work In this work, we introduced GM-OT, a novel approach that models acoustic and linguistic sequences as ordered graphs for alignment. By structuring feature embeddings as nodes and capturing sequential relationships as edges, GM-OT enable s a more comprehensive alignment through graph-based optimal transport. This unified framework ...

    work page

  8. [7]

    Listen, attend a nd spell: A neural network for large vocabulary conversational speech recog- nition,

    W. Chan, N. Jaitly, Q. Le and O. Vinyals, “Listen, attend a nd spell: A neural network for large vocabulary conversational speech recog- nition,” in Proc. of ICASSP, pp. 4960-4964, 2016

    work page 2016

Show all 49 references
  1. [8]

    Joint CTC-attention ba sed end- to-end speech recognition using multi-task learning,

    S. Kim, T. Hori, and S. Watanabe, “Joint CTC-attention ba sed end- to-end speech recognition using multi-task learning,” in Proc. of ICASSP, pp. 4835–4839, 2017

    2017

  2. [9]

    Joint ctc/atten tion decod- ing for end-to-end speech recognition,

    T. Hori, S. Watanabe, and J. R. Hershey, “Joint ctc/atten tion decod- ing for end-to-end speech recognition,” in Proc. of ACL, vol. 1, pp. 518–529, 2017

    2017

  3. [10]

    Hy- brid CTC/Attention Architecture for End-to-End Speech Rec ogni- tion,

    S. Watanabe, T. Hori, S. Kim, J. R. Hershey and T. Hayashi, “Hy- brid CTC/Attention Architecture for End-to-End Speech Rec ogni- tion,” IEEE Journal of Selected Topics in Signal Processing , vol. 11, no. 8, pp. 1240-1253, 2017

    2017

  4. [11]

    Sequence transduction with recurrent neura l net- works,

    A. Graves, “Sequence transduction with recurrent neura l net- works,” arXiv preprint, arXiv:1211.3711, 2012

    2012 arXiv

  5. [12]

    Effective sentence scoring method using BERT for speech recognition,

    J. Shin, Y . Lee, and K. Jung, “Effective sentence scoring method using BERT for speech recognition,” in Proc. of ACML, pp. 1081- 1093, 2019

    2019

  6. [13]

    Masked La nguage Model Scoring,

    J. Salazar, D. Liang, T. Nguyen, K. Kirchhoff, “Masked La nguage Model Scoring,” in Proc. of ACL, pp. 2699-2712, 2020

    2020

  7. [14]

    Towards end to-end speech reco gnition with recurrent neural networks,

    A. Graves, and N. Jaitly, “Towards end to-end speech reco gnition with recurrent neural networks,” in Proc. ICML , pp. 1764–1772, 2014

    2014

  8. [15]

    Hierarchical cond itional end-to-end asr with ctc and multi-granular subword units,

    Higuchi, K. Karube, T. Ogawa, et al., “Hierarchical cond itional end-to-end asr with ctc and multi-granular subword units,” in Proc. of ICASSP, pp. 7797-7801, 2022

    2022

  9. [16]

    Multi-sequence intermediate conditioning for ctc-based asr,

    Y . Fujita, T. Komatsu, and Y . Kida, “Multi-sequence intermediate conditioning for ctc-based asr,” arXiv preprint, arXiv:2204.00175, 2022

    2022 arXiv

  10. [17]

    Bert: Pre train- ing of deep bidirectional transformers for language understanding,

    J. Devlin, M. Chang, K. Lee, and K. Toutanova, “Bert: Pre train- ing of deep bidirectional transformers for language understanding,” arXiv preprint, arXiv:1810.04805, 2018

    2018 arXiv

  11. [18]

    Y . Bai, J. Yi, J. Tao, Z. Tian, Z. Wen and S. Zhang, ”Fast En d- to-End Speech Recognition Via Non-Autoregressive Models a nd Cross-Modal Knowledge Transferring From BERT,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1897-1911, 2021

    1911

  12. [19]

    Non-autoregressive ASR Model ing using Pre-trained Language Models for Chinese Speech Recog ni- tion,

    F. Y u, K. Chen, and K. Lu, “Non-autoregressive ASR Model ing using Pre-trained Language Models for Chinese Speech Recog ni- tion,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 1474-1482, 2022

    2022

  13. [20]

    Knowledge Transfer f rom Large-Scale Pretrained Language Models to End-To-End Spee ch Recognizers,

    Y . Kubo, S. Karita, M. Bacchiani, “Knowledge Transfer f rom Large-Scale Pretrained Language Models to End-To-End Spee ch Recognizers,” in Proc. of ICASSP, pp. 8512-8516, 2022

    2022

  14. [21]

    Distilling a Pretrained Language Mod el to a Multilingual ASR Model,

    K. Choi, H. Park, “Distilling a Pretrained Language Mod el to a Multilingual ASR Model,” in Proc. of INTERSPEECH , pp. 2203- 2207, 2022

    2022

  15. [22]

    Wav2vec 2.0: A framework for self-supervised learning of speech represen tations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “Wav2vec 2.0: A framework for self-supervised learning of speech represen tations,” in Proc. of NeurIPS, 2020

    2020

  16. [23]

    Distilling the Knowledge of BERT for CTC-based ASR,

    H. Futami, H. Inaguma, M. Mimura, S. Sakai, T. Kawahara, “Distilling the Knowledge of BERT for CTC-based ASR,” CoRR abs/2209.02030, 2022

    2022 arXiv

  17. [24]

    BEC- TRA: Transducer-Based End-To-End ASR with Bert-Enhanced Encoder,

    Y . Higuchi, T. Ogawa, T. Kobayashi, S. Watanabe, “BEC- TRA: Transducer-Based End-To-End ASR with Bert-Enhanced Encoder,” in Proc. of ICASSP, pp. 1-5, 2023

    2023

  18. [25]

    Improving End-to-End Contextual Speech Recognition with Fine- Grained Contextual Knowledge Selection,

    M. Han, L. Dong, Z. Liang, M. Cai, S. Zhou, Z. Ma, B. Xu, “Improving End-to-End Contextual Speech Recognition with Fine- Grained Contextual Knowledge Selection,” in Proc. of ICASSP, pp. 8532-8536, 2022

    2022

  19. [26]

    Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation,

    M. Han, F. Chen, J. Shi, S. Xu, B. Xu, “Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation,” arXiv preprint, arXiv:2301.13003, 2023

    2023 arXiv

  20. [27]

    A Context-aware Knowledge Transferr ing Strategy for CTC-based ASR,

    K. Lu and K. Chen, “A Context-aware Knowledge Transferr ing Strategy for CTC-based ASR,” in Proc. of SLT, pp. 60-67, 2022

    2022

  21. [29]

    Improving Non-Autoregressive End-to-End Speech Recogni tion with Pre-Trained Acoustic and Language Models,

    K. Deng, Z. Y ang, S. Watanabe, Y . Higuchi, G. Cheng, P . Zh ang, “Improving Non-Autoregressive End-to-End Speech Recogni tion with Pre-Trained Acoustic and Language Models,” in Proc. of ICASSP, pp. 8522-8526, 2022

    2022

  22. [30]

    Attention is all you ne ed,

    A. V aswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jone s, A. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you ne ed,” in Proc. of NIPS, pp. 5998-6008, 2017

    2017

  23. [31]

    Villani, Optimal transport: old and new, volume 338

    C. Villani, Optimal transport: old and new, volume 338. Springer, 2009

    2009

  24. [32]

    J oint distribution optimal transportation for domain adaptation,

    N. Courty, R. Flamary, A. Habrard, A. Rakotomamonjy, “J oint distribution optimal transportation for domain adaptation,” in Proc. of NIPS, pp. 3733-3742, 2017

    2017

  25. [33]

    Unsupervised Neural A dap- tation Model Based on Optimal Transport for Spoken Language Identification,

    X. Lu, P . Shen, Y . Tsao, H. Kawai, “Unsupervised Neural A dap- tation Model Based on Optimal Transport for Spoken Language Identification,” in Proc. of ICASSP, pp. 7213-7217, 2021

    2021

  26. [34]

    Cross-Modal Alignmen t with Optimal Transport for CTC-Based ASR,

    X. Lu, P . Shen, Y . Tsao, H. Kawai, “Cross-Modal Alignmen t with Optimal Transport for CTC-Based ASR,” IEEE-ASRU, 2023, Dec.16-20,Taipei, Taiwan

    2023

  27. [35]

    Temporal Order Pre- served Optimal Transport-Based Cross-Modal Knowledge Transfer Learning for ASR,

    X. Lu, P . Shen, Y . Tsao, H. Kawai, “Temporal Order Pre- served Optimal Transport-Based Cross-Modal Knowledge Transfer Learning for ASR,” IEEE-SLT, 2024, pp.1-8,Macao, China

    2024

  28. [36]

    Unsupervised Noise Ada p- tive Speech Enhancement by Discriminator-Constrained Opt imal Transport,

    H. Lin, H. Tseng, X. Lu, Y . Tsao, “Unsupervised Noise Ada p- tive Speech Enhancement by Discriminator-Constrained Opt imal Transport,” in Proc. of NeurIPS, pp. 19935-19946, 2021

    2021

  29. [37]

    Interpretation s of Do- main Adaptations via Layer V ariational Analysis,

    H. Tseng, H. Lin, H. Hsuan, and Y . Tsao, “Interpretation s of Do- main Adaptations via Layer V ariational Analysis,” arXiv preprint, CoRR abs/2302.01798, 2023

    2023 arXiv

  30. [38]

    Optimizing Alignment of Speech and Language Latent Spaces for End-To-End Speech Recognition and Understanding,

    W. Wang, S. Ren, Y . Qian, S. Liu, Y . Shi, Y . Qian, M. Zeng, “Optimizing Alignment of Speech and Language Latent Spaces for End-To-End Speech Recognition and Understanding,” in Proc. of ICASSP, pp. 7802-7806, 2021

    2021

  31. [39]

    CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation,

    Y . Zhou, Q. Fang, Y . Feng, “CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation,” arXiv preprint , arXiv:2305.14635, 2023

    2023 arXiv

  32. [40]

    Pre- training for Speech Translation: CTC Meets Optimal Transpo rt,

    P . Le, H. Gong, C. Wang, J. Pino, B. Lecouteux, D. Schwab, “Pre- training for Speech Translation: CTC Meets Optimal Transpo rt,” arXiv preprint, CoRR abs/2301.11716, 2023

    2023 arXiv

  33. [41]

    Conformer: Convolut ion augmented transformer for speech recognition,

    A. Gulati, J. Qin, C. Chiu, et al., “Conformer: Convolut ion augmented transformer for speech recognition,” arXiv preprint , arXiv:2005.08100, 2020

    2005 arXiv

  34. [42]

    Sinkhorn distances: Lightspeed computati on of opti- mal transport,

    M. Cuturi, “Sinkhorn distances: Lightspeed computati on of opti- mal transport,” in Proc. of NIPS, vol. 26, 2013

    2013

  35. [43]

    Gromov-Wasserstein A verag- ing of Kernel and Distance Matrices,

    G. Peyre, M. Cuturi, J. Solomon, “Gromov-Wasserstein A verag- ing of Kernel and Distance Matrices,” Proc. of ICML , vol. 48, pp. 2664-2672, 2016

    2016

  36. [44]

    Fused Gromov-Wasserstein Distance for Structured Objects,

    T. V ayer, L. Chapel, R. Flamary, R. Tavenard, N. Courty, “Fused Gromov-Wasserstein Distance for Structured Objects,”Algorithms, vol. 13, no. 9, 212, 220. https://doi.org/10.3390/a130902 12

    doi

  37. [45]

    A fast proximal point method for Wasserstein distance,

    Y . Xie, X. Wang, R. Wang, H. Zha, “A fast proximal point method for Wasserstein distance,” arXiv preprint arXiv:1802.04307, 2018

    2018 arXiv

  38. [46]

    AIShell-1: An open-source mandarin speech corpus and a spe ech recognition baseline,

    Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, and Hao Zheng, “AIShell-1: An open-source mandarin speech corpus and a spe ech recognition baseline,” in Proc. of COCOSDA, pp. 1-5, 2017

    2017

  39. [47]

    https://huggingface.co/

  40. [48]

    Adam: A Method for Stocha stic Optimization,

    Diederik P . Kingma, Jimmy Ba, “Adam: A Method for Stocha stic Optimization,” in Proc. of ICLR, 2015

    2015

  41. [49]

    WeNet 2.0: More Productive End-to- End Speech Recognition Toolkit,

    B. Zhang, D. Wu, Z. Peng, X. Song, Z. Y ao, H. Lv, L. Xie, C. Y ang, F. Pan, J. Niu, “WeNet 2.0: More Productive End-to- End Speech Recognition Toolkit,” in Proc. of INTERSPEECH, pp. 1661-1665, 2022

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.