Pith. sign in

REVIEW 3 major objections 6 minor 42 references

Denoising GER: A Noise-Robust Generative Error Correction with LLM for Speech Recognition

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims Denoising GER, a three-part framework around a frozen LLM, lowers WER on noisy ASR post-processing and generalizes to unseen noise and clean speech.

desk verdict Ground-truth transcript used to compute fusion weights with no inference-time recipe; Table IV also contradicts the 'all components help' narrative. read the letter →

arxiv 2509.04392 v1 pith:LZJAKGVS submitted 2025-09-04 cs.SD

Yanyan Liu , Minqiang Xu , Yihao Chen , Liang He , Lei Fang , Sian Fang , Lin Liu This is my paper
classification cs.SD
keywords automaticspeechrecognitiongenerativeerrorcorrectionlargelanguagemodelsmulti-modalfusionnoiserobustnessreinforcementlearningwordrateacousticadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generative error correction (GER) for speech recognition post-processes an ASR system's n-best hypotheses with a large language model to produce a better transcription. This paper argues that GER can be made noise-robust by adding three components to a frozen LLM: a noise-adaptive acoustic encoder that cleans the acoustic features before they reach a frozen Whisper encoder, a heterogeneous-feature compensation dynamic fusion (HFCDF) mechanism that reconciles speech and text embeddings before the LLM sees them, and a reinforcement-learning loss that directly minimizes word error rate. The authors show WER reductions on synthesized noisy LibriSpeech, real noisy CHIME4, unseen MUSAN noise, and clean test sets, with the full framework beating text-only GER and prior multi-modal GER baselines. If the result holds, it means noisy ASR can be corrected more accurately without retraining the large model and without sacrificing clean-speech performance.

What carries the argument

The load-bearing machinery is the three-component training objective and the fusion identity inside HFCDF. NAAE is a U-Net adapter on the input spectrogram: X_in = X + adapter_θ(X_in), trained with a weighted cross-entropy plus L1 reconstruction loss, so the frozen ASR encoder sees noise-adapted features and yields dynamic n-best hypotheses. HFCDF uses the modality difference vectors Δx = X_tok − Y_tok and Δy = Y_tok − X_tok to form compensated features, then computes fusion weight µ as softmax over cosine similarities between each compensated feature and the target output; the final multimodal input is Concat(µX'_tok, (1−µ)Y'_tok−top1). RL-loss is the minimum-WER policy-gradient term L_RL =

What would settle it

A concrete check: rerun Table I with HFCDF weights computed from the ASR top-1 hypothesis instead of the ground-truth y. If WER is unchanged, the target-dependence is benign; if WER rises, the HFCDF gains as reported depend on information that is not available in real inference.

Watch

Extended reading notes

Core claim

The paper's central claim is that Denoising GER—the combination of NAAE, HFCDF, and RL-loss—achieves the best WER on noisy speech while retaining accuracy on clean speech. NAAE inserts a small trainable U-Net adapter before a frozen Whisper-Large v2 encoder; the adapter transforms noisy Mel spectrograms back toward a clean-like representation and lets the ASR decoder emit a dynamic n-best list instead of a fixed one. HFCDF then computes difference vectors between acoustic and text embeddings, applies cross-modal compensation, and weights the two compensated streams by softmax cosine similarity to the target. The LLM (Qwen-7B, LoRA-tuned) receives the fused frame-level multimodal representati

Load-bearing premise

The HFCDF fusion weights are defined using the ground-truth transcription y, and the paper does not state what value is used at inference when y is unknown; the reported test-time gains therefore rest on an unspecified stand-in for the target.

Editorial extensions

If this is right

  • Noisy ASR post-processing can be improved by adapting the acoustic front end; the full framework needs only 26M trainable parameters around a frozen 7B LLM.
  • Dynamic, similarity-weighted fusion of acoustic and text streams can outperform fixed concatenation because it reduces the speech-text modality gap.
  • Adding a minimum-WER reinforcement objective to cross-entropy improves sentence-level correction in noise without hurting character-level training.
  • The noise adaptation transfers: gains on in-domain noisy data persist on unseen MUSAN noise and on clean LibriSpeech, so robustness does not trade away clean accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the HFCDF fusion weight is defined through the ground-truth transcription y, a test-time implementation needs a substitute reference (for example, the ASR 1-best hypothesis); the paper does not specify one, and checking whether the gains survive this substitution is a direct way to test the mechanism.
  • The NAAE input-reconstruction idea is a general learned denoising front end; it could be applied to other ASR post-processing tasks or to streaming ASR if the U-Net adapter is made causal.
  • The same difference-vector compensation plus similarity weighting could be transferred to other paired modalities, such as audio-visual speech, where one modality is noisier than the other.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Denoising GER, a multi-modal generative error correction framework for noisy ASR. It combines a noise-adaptive acoustic encoder (NAAE), a heterogeneous feature compensation dynamic fusion (HFCDF) mechanism, and a minimum-WER reinforcement learning loss (RL-loss) to improve LLM-based error correction. The authors report WER improvements on synthetic and real noisy datasets (DNS LibriSpeech, CHiME-4), unseen noise (MUSAN), and clean LibriSpeech, and present ablations supporting each component. However, the HFCDF mechanism computes dynamic fusion weights using the ground-truth target transcription y, with no inference-time substitute, so the method as described is not deployable and the reported gains may reflect label leakage. Additionally, Table IV shows the full model is worse than an ablation on CHiME-4 test-real, contradicting the claim that all components together give the best result.

Significance. If the method were sound, it would be a relevant contribution to LLM-based ASR post-processing, showing how to adapt acoustic encoders, fuse modalities, and optimize WER with RL. The paper has some commendable features: it uses public models (Whisper-Large v2, Qwen-7B), keeps trainable parameters small (26M), and includes experiments on in-domain, out-of-domain, and clean speech. However, the central HFCDF mechanism is oracle-dependent, which invalidates the main technical contribution and the empirical claims in Tables I and III. The inconsistency in Table IV further undermines the component-wise claims. The significance of the work cannot be assessed until the fusion mechanism is redefined and results are re-obtained without using the reference transcript.

major comments (3)
  1. [§III-D, Eqs. (11)–(13)] The dynamic fusion weights μ are computed as softmax over cosine similarities between each modality feature (X′tok, Y′tok) and y, where y is defined as the target output (i.e., ground-truth transcription). At inference time, the ground-truth y is unavailable, and the paper supplies no replacement (e.g., the 1-best hypothesis, the ASR output, or a learned estimator). As written, the method cannot be run on test data; if the reference was used during evaluation, the reported WERs in Tables I and III are not achievable in practice and constitute label leakage. This is a load-bearing flaw, not a minor implementation detail, because Table III credits HFCDF's gain over Concat/Transformer fusion to these oracle-derived weights. The authors must specify an inference-time substitute and rerun all experiments; otherwise the central claims are unverifiable.
  2. [Table IV] Table IV directly contradicts the claim that combining all components yields the best result. On CHiME-4 test-real, the full model (NAAE + HFCDF + RL-loss) gives WER 5.32, while RL-loss alone gives 4.92 and NAAE+HFCDF gives 4.75. The text states that adding RL-loss enhances performance and that the full model is optimal, but the table shows a clear degradation on a real noisy test set after adding RL-loss to NAAE+HFCDF. This inconsistency must be resolved: either the table is wrong, or the claim that the complete framework improves robustness is unsupported. The ablation conclusions in Section V-B need to be revised with a correct and consistent set of results.
  3. [§III-B, Eqs. (3)–(5)] The notation in the NAAE formulation is ambiguous and inconsistent, which undermines reproducibility. Eq. (3) writes Xin = fθ(Xin) = X + adapterθ(Xin), but X is undefined and Xin appears on both sides with different roles. Eq. (5) uses L1(X′in, Xin) where X′in is introduced without definition, and λ is not specified in the experimental setup (only α and β are given). Since NAAE is one of the three main contributions, the objective being optimized must be stated unambiguously, with all variables defined and all hyperparameters specified.
minor comments (6)
  1. [§IV-A] Typo: "CHINE4" should be "CHiME-4".
  2. [Abstract / §I Contributions] The contribution list says "speech-text multi-modal featre HFCDF mechanism" — 'featre' appears to be a typo for 'feature'.
  3. [§V-C] The error analysis cites "UADF [12] and MMGER [14]" but the reference numbering does not match the bibliography entries; in the related work, MMGER is [16]. Please harmonize citations.
  4. [§III-E, Eq. (15)] The RL loss uses w(yi, y∗) but y∗ is not explicitly defined; it later appears as the ground-truth. Define y∗ consistently with y used in Eq. (11), and note whether the same ground-truth is used for the fusion weights during training.
  5. [Figure 1] The figure contains typos: "Languege" and "Embing" should be "Language" and "Embedding". Also, the U-Net adapter structure is not clearly visible in the captioned subfigures; please improve the figure quality.
  6. [§III-D] The hyperparameter k in Eq. (10) is introduced but its value is not reported in Section IV-B. Experimental reproducibility would benefit from listing k along with α and β.

Circularity Check

1 steps flagged · score 6.0 of 10

HFCDF fusion weights are computed from the ground-truth target output y (Eqs. 11–12), so the predicted transcription is conditioned on the answer; no inference-time substitute is specified.

  1. self definitional [Section III-D (Multi-modal Heterogeneous Feature Compensation Dynamic Fusion Mechanism), Equations (11)–(14)]
    "we calculate the similarity scores between the acoustic modality, the text modality, and the target output, and use the Softmax function to convert these scores into weights. ... Ra = Cosine(X'_tok, y); Rt = Cosine(Y'_tok, y) (11); µ = e^{Ra}/(e^{Ra}+e^{Rt}) (12); X'_mmc = Concat(µX'_tok, (1-µ)Y'_tok−top1) (13); P(Y) = Π P(yt|Concat(X'_mmc, Y'_tok), θ1) (14). Here, y represents the target output."

    The fusion weight µ in Eq. (12) is a softmax over cosine similarities between each modality representation and y, the ground-truth target transcript. This µ is then used in Eq. (13) to construct the multi-modal input X'_mmc, and Eq. (14) predicts Y conditional on that input. Thus the model's prediction is conditioned on the very variable it is supposed to predict: the input representation is defined using the answer y. At inference time y is unavailable, and the paper specifies no replacement (e.g., a first-pass hypothesis or learned estimator). Consequently the HFCDF results in Table III and the full-model results in Table IV are not achievable as genuine predictions; they reduce to an oracle-dependent construction. This is not merely a missing implementation detail, because the claimed a

full rationale

The central circular step is the HFCDF mechanism. The paper explicitly defines the fusion weights via cosine similarity with 'the target output' y (Eqs. 11–12), then uses those weights to build the input for the generative model that outputs Y (Eqs. 13–14). This is self-definitional: the prediction input is a function of the prediction target. No training/inference distinction or proxy for y is given, so the method as written cannot run on test data. This taints the main claimed contribution and any table entry involving HFCDF. The NAAE module (Eqs. 3–5) and the RL loss (Eq. 15) are standard supervised/RL objectives using ground truth only as training targets, which is not circular. I found no load-bearing self-citations or imported uniqueness theorems; the paper cites prior GER work (e.g., MMGER, RobustGER) as inspiration and baselines, not as a substitute for its own derivation. There is also an internal inconsistency in Table IV (full model 5.32% vs RL-loss-only 4.92% on CHiME-4 test-real) that undercuts the 'all components together yield the best result' claim, but that is a correctness/consistency issue rather than a circularity. Given that one core component's reported gains reduce by construction to oracle access to y, while other components retain independent content, a score of 6 is appropriate.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The framework relies on a handful of hand-chosen hyperparameters and on the availability of ground-truth text at inference for the fusion weights. No new physical or conceptual entities are postulated.

free parameters (3)
  • k (cross-modal compensation coefficient) = 0.7
    Set by hand (Section IV.B); controls how much acoustic vs text features are compensated. No sensitivity analysis is reported.
  • alpha and beta (loss weighting coefficients) = alpha=0.2, beta=0.2
    Hand-chosen in Eq. 16; no grid search or ablations are shown.
  • lambda (ASR loss interpolation in Eq. 5) = unspecified
    The paper defines lambda in Eq. 5 but never reports its value or selection procedure; an unstated hand-chosen constant.
assumptions (3)
  • ad hoc to paper The dynamic fusion weight mu is computable at inference from cosine similarity with the target output y.
    Section III-D, Eq. 11-12. Without ground truth at test time, mu cannot be computed; no replacement is specified.
  • domain assumption Whisper-Large v2 and Qwen-7B are suitable frozen base models for the GER task.
    Section IV.B; standard choices but their pretraining distribution affects the result.
  • standard math Minimum word error rate loss (Eq. 15) provides a valid reward signal for sequence training.
    Well-established RL objective used in prior ASR work, e.g., SEED-ASR [37].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Denoising GER: A Noise-Robust Generative Error Correction with LLM for Speech Recognition." pith.science (2026). https://pith.science/paper/LZJAKGVS

@misc{pith2026250904392,
  author       = {Pith},
  title        = {Pith review of: Denoising GER: A Noise-Robust Generative Error Correction with LLM for Speech Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LZJAKGVS}},
  note         = {Machine review of arXiv:2509.04392}
}
read the original abstract

In recent years, large language models (LLM) have made significant progress in the task of generation error correction (GER) for automatic speech recognition (ASR) post-processing. However, in complex noisy environments, they still face challenges such as poor adaptability and low information utilization, resulting in limited effectiveness of GER. To address these issues, this paper proposes a noise-robust multi-modal GER framework (Denoising GER). The framework enhances the model's adaptability to different noisy scenarios through a noise-adaptive acoustic encoder and optimizes the integration of multi-modal information via a heterogeneous feature compensation dynamic fusion (HFCDF) mechanism, improving the LLM's utilization of multi-modal information. Additionally, reinforcement learning (RL) training strategies are introduced to enhance the model's predictive capabilities. Experimental results demonstrate that Denoising GER significantly improves accuracy and robustness in noisy environments and exhibits good generalization abilities in unseen noise scenarios.

Figures

Figures reproduced from arXiv: 2509.04392 by the authors.

Figure 1
Figure 1. the overall framework of our proposed Denoising GER with LLM for ASR, Part (a) of Figure 1 represents the adaptive [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 31 canonical work pages

  1. [1]

    Automatic channel selection and spatial feature integration for multi-channel speech recog- nition across various array topologies,

    B. Mu, P. Guo, D. Guo, P. Zhou, W. Chen, and L. Xie, “Automatic channel selection and spatial feature integration for multi-channel speech recog- nition across various array topologies,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 11396–11400, IEEE, 2024

    work page 2024

  2. [2]

    Deliberation model based two-pass end-to-end speech recognition,

    K. Hu, T. N. Sainath, R. Pang, and R. Prabhavalkar, “Deliberation model based two-pass end-to-end speech recognition,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 7799–7803, IEEE, 2020

    work page 2020

  3. [3]

    An analysis of incorporating an external language model into a sequence-to-sequence model,

    A. Kannan, Y . Wu, P. Nguyen, T. N. Sainath, Z. Chen, and R. Prab- havalkar, “An analysis of incorporating an external language model into a sequence-to-sequence model,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 1–5828, IEEE, 2018

    work page 2018

  4. [4]

    Component fusion: Learning replaceable language model component for end-to-end speech recognition system,

    C. Shan, C. Weng, G. Wang, D. Su, M. Luo, D. Yu, and L. Xie, “Component fusion: Learning replaceable language model component for end-to-end speech recognition system,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 5361–5635, IEEE, 2019

    work page 2019

  5. [5]

    Fastcorrect: Fast error correction with edit alignment for automatic speech recognition,

    Y . Leng, X. Tan, L. Zhu, J. Xu, R. Luo, L. Liu, T. Qin, and Li, “Fastcorrect: Fast error correction with edit alignment for automatic speech recognition,” Advances in Neural Information Processing Systems, vol. 34, pp. 21708–21719, 2021

    work page 2021

  6. [6]

    Fastcorrect: Fast error correction with edit alignment for automatic speech recognition,

    Y . Leng, X. Tan, L. Zhu, J. Xu, R. Luo, L. Liu, and Qin, “Fastcorrect: Fast error correction with edit alignment for automatic speech recognition,” Advances in Neural Information Processing Systems , vol. 34, pp. 21708– 21719, 2021

    work page 2021

  7. [7]

    Improving readability for automatic speech recognition tran- scription,

    J. Liao, S. Eskimez, L. Lu, Y . Shi, M. Gong, L. Shou, H. Qu, and M. Zeng, “Improving readability for automatic speech recognition tran- scription,” ACM Transactions on Asian and Low-Resource Language Information Processing, vol. 22, no. 5, pp. 1–23, 2023

    work page 2023

  8. [8]

    N-best t5: Robust asr error correction using multiple input hypotheses and constrained decoding space,

    R. Ma, M. J. Gales, K. M. Knill, and M. Qian, “N-best t5: Robust asr error correction using multiple input hypotheses and constrained decoding space,” in Proc. INTERSPEECH, 2023

    work page 2023

Show all 42 references
  1. [9]

    Towards interfacing large language models with asr systems using confidence measures and prompting,

    M. Naderi, E. Hermann, A. Nanchen, S. Hovsepyan, and M. M. Doss, “Towards interfacing large language models with asr systems using confidence measures and prompting,” in Proc. INTERSPEECH, 2024

    2024

  2. [10]

    Hyporadise: An open baseline for generative speech recognition with large language models,

    C. Chen, Y . Hu, C.-H. H. Yang, S. M. Siniscalchi, P.-Y . Chen, and E.-S. Chng, “Hyporadise: An open baseline for generative speech recognition with large language models,” Advances in Neural Information Processing Systems, vol. 36, pp. 31665–31688, 2023

    2023

  3. [11]

    Noise-robust speech recognition with 10 minutes unparalleled in-domain data,

    C. Chen, N. Hou, Y . Hu, S. Shirol, and E. S. Chng, “Noise-robust speech recognition with 10 minutes unparalleled in-domain data,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4298–4302, IEEE, 2022

    2022

  4. [12]

    It’s never too late: Fusing acoustic information into large language models for automatic speech recognition,

    C. Chen, R. Li, Y . Hu, S. M. Siniscalchi, P.-Y . Chen, E. S. Chng, and C.-H. H. Yang, “It’s never too late: Fusing acoustic information into large language models for automatic speech recognition,” in International Conference on Learning Representations, ICLR. , 2024

    2024

  5. [13]

    Whispering llama: A cross-modal generative error correction framework for speech recognition,

    S. Radhakrishnan, C.-H. Yang, S. Khan, R. Kumar, N. Kiani, D. G ´omez- Cabrero, and J. Tegn ´e, “Whispering llama: A cross-modal generative error correction framework for speech recognition,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proces...

    2023

  6. [14]

    Connecting speech encoder and large language model for asr,

    W. Yu, C. Tang, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Connecting speech encoder and large language model for asr,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 12637–12641, IEEE, 2024

    2024

  7. [15]

    Listen again and choose the right answer: A new paradigm for automatic speech recognition with large language models,

    Y . Hu et al., “Listen again and choose the right answer: A new paradigm for automatic speech recognition with large language models,” in Proc. ACL, 2024

    2024

  8. [16]

    Mmger: Multi-modal and multi-granularity generative error correction with llm for joint accent and speech recognition,

    B. Mu, X. Wan, N. Zheng, H. Zhou, and L. Xie, “Mmger: Multi-modal and multi-granularity generative error correction with llm for joint accent and speech recognition,” IEEE Signal Processing Letters , 2024

    2024

  9. [17]

    Qwen-audio: Advancing universal audio understanding via unified large- scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understanding via unified large- scale audio-language models,” CoRR, vol. abs/2311.07919, 2023

    2023 arXiv

  10. [18]

    An embarrassingly simple approach for LLM with strong ASR capacity,

    Z. Ma, G. Yang, Y . Yang, Z. Gao, J. Wang, Z. Du, F. Yu, Q. Chen, S. Zheng, S. Zhang, and X. Chen, “An embarrassingly simple approach for LLM with strong ASR capacity,” CoRR, vol. abs/2402.08846, 2024

    2024 arXiv

  11. [19]

    Salmonn: Towards generic hearing abilities for large lan- guage models,

    W. Yu, C. Tang, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Salmonn: Towards generic hearing abilities for large lan- guage models,” in International Conference on Learning Representations, ICLR., IEEE, 2024

    2024

  12. [20]

    Prompting large language models with speech recognition abilities,

    Y . Fathullah, C. Wu, E. Lakomkin, J. Jia, Y . Shangguan, K. Li, J. Guo, W. Xiong, J. Mahadeokar, O. Kalinli, et al., “Prompting large language models with speech recognition abilities,” in IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP) , p...

    2024

  13. [21]

    Slm: Bridge the thin gap between speech and text foundation models,

    M. Wang, W. Han, I. Shafran, Z. Wu, C.-C. Chiu, Y . Cao, N. Chen, et al., “Slm: Bridge the thin gap between speech and text foundation models,” in Automatic Speech Recognition and Understanding Workshop (ASRU) , pp. 1–8, IEEE, 2023

    2023

  14. [22]

    Chatting about chatgpt: how may ai and gpt impact academia and libraries?,

    B. D. Lund and T. Wang, “Chatting about chatgpt: how may ai and gpt impact academia and libraries?,” Library hi tech news , vol. 40, no. 3, pp. 26–29, 2023

    2023

  15. [23]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, S. Altenschmidt, S. Anadkat, et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  16. [24]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  17. [25]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” in Proc.ICML, pp. 19730–19742, 2023

    2023

  18. [26]

    Macaw-llm: Multi-modal language modeling with image, audio, video, and text integration,

    C. Lyu, M. Wu, L. Wang, X. Huang, B. Liu, Z. Du, S. Shi, and Z. Tu, “Macaw-llm: Multi-modal language modeling with image, audio, video, and text integration,” arXiv preprint arXiv:2306.09093 , 2023

    2023 arXiv

  19. [27]

    On decoder-only architecture for speech-to-text and large language model integration,

    J. Wu, Y . Gaur, Z. Chen, L. Zhou, Y . Zhu, T. Wang, J. Li, S. Liu, B. Ren, L. Liu, et al. , “On decoder-only architecture for speech-to-text and large language model integration,” in Automatic Speech Recognition and Understanding Workshop (ASRU) , pp. 1–8, IEEE, 2023

    2023

  20. [28]

    Seamless: Multilingual expressive and streaming speech translation,

    L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, M. Dup- penthaler, P.-A. Duquenne, B. Ellis, H. Elsahar, J. Haaheim, et al. , “Seamless: Multilingual expressive and streaming speech translation,” arXiv preprint arXiv:2312.05187 , 2023

    2023 arXiv

  21. [29]

    Au- diopalm: A large language model that can speak and listen,

    P. Rubenstein, C. Asawaroengchai, D. Nguyen, and A. Bapna, “Au- diopalm: A large language model that can speak and listen,” CoRR, vol. abs/2306.12925, 2023

    2023 arXiv

  22. [30]

    Can generative large language models perform asr error correction?,

    R. Ma, M. Qian, P. Manakul, M. Gales, and K. Knill, “Can generative large language models perform asr error correction?,” arXiv preprint arXiv:2307.04172, 2023

    2023 arXiv

  23. [31]

    Robust speech recognition via large-scale weak supervi- sion,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervi- sion,” in International conference on machine learning, pp. 28492–28518, PMLR, 2023

    2023

  24. [32]

    Beats: Audio pre-training with acoustic tokenizers,

    S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, and F. Wei, “Beats: Audio pre-training with acoustic tokenizers,” in Proc. ICML , 2023

    2023

  25. [33]

    Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,” in EMNLP, H. Bouamor, J. Pino, and K. Bali, Eds., 2023

    2023

  26. [34]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5206– 5210, IEEE, 2015

    2015

  27. [35]

    Mrcn: A novel modality restitution and compensation network for visible-infrared person re-identification,

    Y . Zhang, Y . Yan, J. Li, and H. Wang, “Mrcn: A novel modality restitution and compensation network for visible-infrared person re-identification,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, pp. 3498–3506, 2023

    2023

  28. [36]

    Multi-stage auxiliary learning for visible-infrared person re-identification,

    H. Zhang, S. Cheng, and A. Du, “Multi-stage auxiliary learning for visible-infrared person re-identification,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    2024

  29. [37]

    Seed-asr: Understanding diverse speech and contexts with llm-based speech recognition,

    Y . Bai, J. Chen, J. Chen, W. Chen, Z. Chen, C. Ding, L. Dong, Q. Dong, Y . Du, K. Gao, L. Gao, Y . Guo,et al., “Seed-asr: Understanding diverse speech and contexts with llm-based speech recognition,” CoRR, vol. abs/2407.04675, 2024

    2024 arXiv

  30. [38]

    MUSAN: A music, speech, and noise corpus,

    D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,” CoRR, vol. abs/1510.08484, 2015

    2015 arXiv

  31. [39]

    The design for the wall street journal-based csr corpus,

    D. B. Paul and J. Baker, “The design for the wall street journal-based csr corpus,” in Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992 , 1992

    1992

  32. [40]

    The rwth/upb/forth system combination for the 4th chime challenge evaluation,

    T. Menne, “The rwth/upb/forth system combination for the 4th chime challenge evaluation,” 2016

    2016

  33. [41]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, et al. , “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  34. [42]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in International Conference on Learning Representations, ICLR. , 2022

    2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.