Pith. sign in

REVIEW 3 major objections 6 minor 31 references

Whisper-Aware LLM: Self-Supervised Uncertainty Learning for Robust Whispered Speech Recognition

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces the Whisper-Aware LLM, a framework that makes an audio LLM aware of the uncertainty in whispered speech, cutting whisper recognition error by 17% and hallucinations from over 25% to 4.5%.

desk verdict A well-engineered whisper ASR paper whose core uncertainty signal has an undefined F0 target; still valuable, but needs major fixes. read the letter →

arxiv 2608.10836 v1 pith:TYVGCCCR submitted 2026-08-11 cs.SD cs.AI

Gaopeng Xu , Zhenyu Wang , Zheng Xue , Yinfeng Xia , Haitao Yao This is my paper · ORCID
classification cs.SDcs.AI
keywords whisperedspeechrecognitionaudioLLMself-supervisedlearninguncertaintyestimationconfidence-fuseddecodinghallucinationreductionfundamentalfrequency(F0)predictionmaskedspectrumreconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tackles a two-sided failure of speech recognition on whispered speech: models either miss quiet whispered words or invent speech from noise. Its central proposal is that instead of forcing the model to be confident on ambiguous audio, the model should first learn to sense the physical lack of information in the signal. A lightweight uncertainty module is trained with two self-supervised tasks — predicting the pitch contour and reconstructing masked spectrogram frames — without needing whisper transcriptions. The resulting confidence scores then steer an LLM decoder through a global instruction embedding and frame-level attention modulation. On the paper's whispered Mandarin benchmark, the method reports a character error rate of 1.31%, a 17% relative improvement over the previous best system, while hallucination on non-speech noise drops from over 25% to 4.5%.

What carries the argument

The carrying object is the Uncertainty Perception Module (UPM), a small network attached to a frozen acoustic encoder that produces both a global uncertainty vector and a frame-wise confidence sequence. Its primary supervision is fundamental-frequency (F0) contour prediction: the head $\hat F_0$ is trained with mean squared error against a ground-truth contour $F_0$, and the frame confidence is $h_{\mathrm{conf}}(t)=1-\mathrm{clip}(e_{F_0}(t)/\epsilon_{\mathrm{max}},0,1)$, where $e_{F_0}$ is the normalized prediction error. A second head reconstructs randomly masked spectrum frames and its pooled hidden state forms the global vector. The decoder is then controlled through an MLP that turns the global vector into an instruction embedding and through an additive attention bias $w\cdot h_{\mathrm{conf}}(t)$ added to the score for acoustic frame $t$. A three-stage training recipe — UPM pretraining, interface adaptation, then full low-rank fine-tuning — keeps the large decoder stable while the uncertainty signals are learned.

What would settle it

Measure the UPM's confidence output on a controlled set of whispered, voiced, and noise-only audio. If confidence is essentially binary — near 1 on voiced frames and near 0 on all whisper/noise frames — the F0-prediction target is acting as a voicing detector, and a simpler voicing feature should replicate the results. A second check is to feed the model whispered speech with a synthetic pitch contour superimposed; if recognition does not change, the global instruction is not using graded uncertainty.

Watch

Extended reading notes

Core claim

The paper claims that uncertainty can be learned rather than assumed: an audio LLM can be taught to perceive degraded signal quality through self-supervision, and that perception can be used to control generation. The framework estimates two signals from the encoder: a per-frame confidence, derived from how well the model can predict the fundamental-frequency contour, and a global uncertainty vector from masked spectrum reconstruction. These are fused into decoding by prepending a learned instruction embedding and by adding a weighted attention bias to acoustic frames, so the decoder pays less attention to frames it deems unreliable. On the AISHELL6-Whisper benchmark the reported whisper CER is 1.31% versus 1.58% for the previous best system, and on a curated set of noise-only clips the hallucination rate falls from over 25% to 4.5%. The paper also reports that general, non-whispered ASR remains at a competitive level, and its ablation attributes most of the gain to the global instruction rather than the frame-level attention bias.

Load-bearing premise

The load-bearing premise is that a ground-truth fundamental-frequency contour exists and is meaningful for every audio frame used in training, even though whispered speech and noise have no voiced pitch, so the learned 'uncertainty' may end up measuring whether the frame is voiced rather than how ambiguous the signal is.

Editorial extensions

If this is right

  • Whispered speech recognition can be improved on top of a general-purpose audio LLM without adding whisper transcriptions, since the uncertainty targets are self-supervised.
  • The same uncertainty signal that improves accuracy also teaches the model when to stay silent, reducing spurious transcriptions of noise.
  • Adding uncertainty perception does not trade away normal speech ASR: the model stays competitive on clean Mandarin and English benchmarks.
  • The global instruction embedding carries most of the benefit; frame-level attention modulation alone gives a smaller gain, suggesting the decoder uses high-level signal-quality awareness more than per-frame reweighting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because the confidence score is a monotone function of F0-prediction error, the method may be learning a sophisticated voicing detector; a falsifiable check is to compare against a standalone voiced/unvoiced classifier on the same encoder.
  • Editorial extension: the same recipe could transfer to other low-information conditions where a physical feature is missing — whispered consonants, distant microphone speech, or codec-degraded audio — by swapping the F0 target for whatever physical feature is degraded.
  • Editorial extension: the hallucination test set is custom and modest in size; a publicly shared evaluation set would let other groups verify whether the 4.5% rate generalizes outside these clips.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Whisper-Aware LLM, an Audio-LLM extension with an Uncertainty Perception Module trained by two self-supervised tasks: F0 contour prediction and masked spectrum reconstruction. The learned confidence is injected into the LLM decoder through a global instruction embedding and an additive attention bias, with a three-stage training protocol. On AISHELL6-Whisper the model reports 1.31% CER on whispered Chinese (17% relative improvement over Seed-ASR), competitive general ASR results on AISHELL-1 and LibriSpeech, and a reduction of hallucination rate from over 25% to 4.5% on a custom Noise Hallucination Set. The paper argues that the uncertainty signal, rather than the added noise data, drives these gains.

Significance. The framework is conceptually attractive: instead of adding whisper-specific data or static adaptation, it attempts to make the decoder aware of acoustic uncertainty in real time. The architecture is simple, and the ablation in Table 5 indicates that the global-instruction branch is highly effective. The empirical results on public whisper benchmarks are strong and cross-lingual, which would be a useful contribution if the uncertainty mechanism were validated. However, the manuscript does not establish that the F0-prediction target is meaningful for whispered and non-speech audio, which is the foundation of the proposed uncertainty signal; the lack of error bars and of a publicly available hallucination set further limits the strength of the stated conclusions.

major comments (3)
  1. [Section 2.2.1, Eq. (1)-(2)] The ground-truth F0 contour F0(t) used in Eq. (1) is not defined for whispered speech or pure-noise segments, which by definition lack a fundamental frequency. If F0(t) is set to 0 for such frames, then the UPM will learn to predict near-zero values on whisper/noise, making e_F0(t) small and h_conf(t) in Eq. (2) close to 1; the additive bias in Eq. (3) would then upweight the very frames that are least reliable. If instead F0(t) is taken from a pitch tracker applied to whisper/noise, the targets are arbitrary or noisy, and the claim that the task is label-free, self-supervised, and physics-informed is unsupported. Please specify the exact target construction for whispered, unvoiced, and non-speech frames, and provide a diagnostic showing that the trained UPM assigns low confidence precisely where ASR uncertainty is high (e.g., per-condition distributions of h_conf on whisper, normal speech, and noise).
  2. [Table 5 and Section 3.3] The ablation does not isolate the contribution of the two self-supervised tasks, so the reported gains cannot be attributed to the uncertainty signal. The 'Global Instruction Only' configuration already lowers CER to 1.84% from 3.98%, and that instruction is produced from the same UPM hidden state trained with both LF0 and LSpec; the 'Attention Modulation Only' configuration, which uses h_conf directly in Eq. (3), gives a much smaller gain to 3.45%. To support the central claim, the authors should ablate the F0-prediction loss and the masked-reconstruction loss separately (e.g., UPM trained with only LF0, only LSpec, and neither), and report whether h_conf remains predictive of ASR errors when the F0 branch is removed.
  3. [Section 3.1.1 and Tables 1-4] Several evaluation details are missing that are necessary to support the headline numbers. The Noise Hallucination Set is a custom corpus whose construction is not described (sources, noise types, overlap with the 200h noise used in fine-tuning, and whether it will be released), and no error bars are reported on any of the CER/WER/HR numbers; the gaps in Table 2, e.g., 1.31% vs. 1.58% CER, may be within sampling variability. Because the fine-tuning set includes 200h of pure noise, the hallucination-rate drop from over 25% to 4.5% could be due to noise exposure rather than to the uncertainty mechanism; please provide dataset details, release plans, confidence intervals or significance tests, and an ablation that removes the noise subset from fine-tuning.
minor comments (6)
  1. [Section 3.1.3 and Tables 1-2] The row label 'Aishell6-whisper A VSR' should be 'AVSR' (audio-visual speech recognition); please correct the name and the citation to the AISHELL6-Whisper work, and ensure the two different arXiv identifiers cited as [2] and [14] refer to the intended versions.
  2. [References] References [13] and [15] are identical (B. P. Lim, 'Computational differences between whispered and non-whispered speech'); please merge them and update the in-text citations.
  3. [Figure 1] The figure uses unexplained abbreviations ('WI', 'TR', 'SR', 'IP', 'FFN', 'MS'); please add a legend or define them in the caption.
  4. [Section 2.3.2] After computing h_conf(t) in Eq. (2), the text says Hconf is centered to zero mean; please state whether this centering is applied at training and inference and how it interacts with the learnable scalar w, because centering changes the effective bias to w*(h_conf(t) - mean(Hconf)).
  5. [Sections 1 and 2.2] The terms 'label-free' and 'self-supervised' are used for F0 contour prediction, but the regression target F0(t) must be obtained from an external source; please clarify how the target is generated, or rephrase the terminology.
  6. [Section 3.1.1] The composition of the 'pure noise' corpus is not described; please specify the noise types, SNRs, and ensure there is no overlap with the Noise Hallucination Set.

Circularity Check

1 steps flagged · score 4.0 of 10

F0-confidence definition inverts uncertainty for whispered/noise frames; empirical CER/HR results otherwise self-contained.

  1. self definitional [Section 2.2.1, Eqs. (1)-(2), with Section 1's definition of whispered speech]
    "The frame-wise confidence score hconf(t) is derived from the normalized prediction error eF0(t): hconf(t) = 1−clip( eF0(t)/εmax ,0,1). ... Whispered speech ... characterized by the absence of a fundamental frequency (F0) and harmonic structure."

    Eq. (2) makes h_conf a decreasing function of F0 prediction error. The paper's own characterization of whispered speech is 'the absence of a fundamental frequency (F0)'. For whisper/noise frames, the only physically meaningful F0 target is 0 (undefined/absent), so after training the UPM predicts ≈0, e_F0≈0, and h_conf≈1. Thus the model is maximally confident exactly on the unreliable frames Eq. (3) is supposed to downweight. The 'learned uncertainty' is the negation of uncertainty by construction; the attention-modulation story holds only if an unstated, arbitrary pitch-tracker target is used, in which case the 'physics-informed, self-supervised' derivation is unsupported.

full rationale

The paper's final empirical claims (CER 1.31% on AISHELL6-Whisper, HR 4.5% on the Noise Hallucination Set) are standard supervised evaluations on held-out benchmarks; they do not reduce to a fitted parameter or a self-citation chain. There are no load-bearing self-citations: references [1], [2], [15] are external, and the Qwen2-Audio base model is used as a system component, not as proof of the uncertainty mechanism. The one definitional problem is in the UPM confidence construction: confidence is defined as 1 minus F0-prediction error, while whispered/noise audio is defined by F0 absence. Consequently, a well-trained UPM yields near-zero error and near-one confidence for the very frames that should be flagged as uncertain; the claimed uncertainty signal is the opposite of uncertainty by construction. This is a self-definitional issue in the core mechanism, but it does not make the reported CER/HR numbers circular, since those numbers could still be driven by other components (e.g., the global instruction or noise training data). Score 4 reflects a central mechanism that is partly definitionally flawed while the central empirical claims retain independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the F0-absence assumption, the proxy validity of the SSL losses, and the availability of F0 targets for whisper/noise. These are domain or ad hoc assumptions, not independently established. The free parameters are mainly training hyperparameters that shape the confidence signal.

free parameters (3)
  • epsilon_max = not specified
    Clipping threshold in Eq. (2), hand-chosen, converts F0 prediction error into confidence.
  • lambda_aux = 0.1
    Weight for auxiliary losses in Stage 3 composite loss.
  • w = learned
    Attention modulation scalar in Eq. (3), fitted during training.
assumptions (4)
  • domain assumption The lack of glottal information (F0) is the single most critical factor degrading whispered speech recognition.
    Invoked in the Introduction from [1] to motivate the F0 prediction task; if false, the task's relevance is weakened.
  • ad hoc to paper F0 prediction error and masked spectrum reconstruction loss are valid proxies for acoustic signal uncertainty.
    The paper provides no independent validation that these losses correspond to the uncertainty that drives hallucination.
  • ad hoc to paper Ground-truth F0 contours exist for all training audio, including whispered speech and pure noise.
    The F0 target in Eq. (1) is undefined for whispers and noise, yet used as supervision without discussion.
  • domain assumption Additive attention bias with a single learnable scalar w can steer the LLM decoder's reliance on acoustic frames.
    Standard attention-modulation assumption; not proven for this architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Whisper-Aware LLM: Self-Supervised Uncertainty Learning for Robust Whispered Speech Recognition." pith.science (2026). https://pith.science/paper/TYVGCCCR

@misc{pith2026260810836,
  author       = {Pith},
  title        = {Pith review of: Whisper-Aware LLM: Self-Supervised Uncertainty Learning for Robust Whispered Speech Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TYVGCCCR}},
  note         = {Machine review of arXiv:2608.10836}
}
read the original abstract

The signal ambiguity of whispered speech drives ASR systems toward two opposing failure modes: failing to capture whispered speech or hallucinatory transcription of noise. This paper introduces the Whisper-Aware LLM, a framework that teaches an Audio-LLM to perceive and react to this uncertainty. Our model develops an intrinsic self-awareness by learning to quantify the physical deficiencies of acoustic signals through targeted self-supervised tasks. This learned uncertainty is then operationalized via a novel Confidence-Fused Decoding mechanism, which provides both high-level instructions and frame-level attention modulation to the LLM decoder. Our experiments confirm the effectiveness of this approach. The model sets a new state-of-the-art on whispered speech with a 17% relative CER reduction on AISHELL6-Whisper. At the same time, it directly addresses the reliability trade-off, with hallucination rates dropping from over 25% to 4.5%.

Figures

Figures reproduced from arXiv: 2608.10836 by the authors.

Figure 1
Figure 1. The overall architecture of the Whisper-Aware LLM. The model first encodes the audio into acoustic representations. Our UPM then processes these representations to perceive signal quality, producing a global instruction embedding (einstruct) and a frame￾wise confidence sequence (Hconf). These outputs guide the LLM decoder: the instruction acts as a system prompt, and the confidence scores modulate the attention mech… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 15 canonical work pages

  1. [1]

    pseudo- whisper

    Introduction Whispered speech, a mode of phonation without vocal fold vi- bration, presents a formidable and long-standing challenge for Automatic Speech Recognition (ASR) systems. The acoustic manifestation of whispering characterized by the absence of a fundamental frequency (F0) and harmonic structure, and a noise-like quality [1, 2] creates a fundamen...

    work page

  2. [2]

    A novel framework integrating self-supervised uncertainty perception directly into an Audio-LLM

    work page

  3. [3]

    Two physics-informed self-supervised tasks for label-free uncertainty learning

    work page

  4. [4]

    A Confidence-Fused Decoding mechanism for dynamic gen- erative control

    work page

  5. [5]

    A systematic three-stage training protocol for robust repre- sentation learning

    work page

  6. [6]

    The Whisper-Aware LLM Framework Our framework enhances a standard Audio-LLM by integrat- ing a lightweight Uncertainty Perception Module (UPM) and a corresponding Confidence-Fused Decoding mechanism. 2.1. Overall Architecture As shown in Fig.1, the architecture consists of three main stages: •Acoustic Encoding:The encoderEmaps an input au- dio waveformwin...

    work page Pith review arXiv 2026

  7. [7]

    Experiments To validate the effectiveness of our Whisper-Aware LLM, we conducted a series of comprehensive experiments. We aimed to evaluate its performance on whispered speech recognition, its general ASR capabilities, its robustness against hallucination, and the contribution of its core components. 3.1. Experimental Setup 3.1.1. Datasets and Metrics Tr...

    work page

  8. [8]

    Seed-asr: Understanding diverse speech and contexts with llm-based speech recognition,

    Y . Bai, J. Chen, J. Chen, W. Chen, Z. Chen, C. Ding, L. Dong, Q. Dong, Y . Du, K. Gaoet al., “Seed-asr: Understanding diverse speech and contexts with llm-based speech recognition,”arXiv preprint arXiv:2407.04675, 2024

    arXiv 2024

Show all 31 references
  1. [9]

    On the AISHELL-1 test set, it achieves a strong CER of 1.34%, placing it on par with other top-tier systems like Funasr (1.22%)

    Our model demonstrates highly competitive performance against state-of-the-art systems. On the AISHELL-1 test set, it achieves a strong CER of 1.34%, placing it on par with other top-tier systems like Funasr (1.22%). On the LibriSpeech-clean set, our model obtains a robust WER...

  2. [10]

    Instead of just matching patterns, our approach teaches the model to first perceive signal quality through targeted self- supervised tasks

    Conclusion In this paper, we introduced the Whisper-Aware LLM, a frame- work designed to handle the uncertainty inherent in whispered speech. Instead of just matching patterns, our approach teaches the model to first perceive signal quality through targeted self- supervised ta...

  3. [11]

    All technical content, methodology, experimental design, and analysis were conducted entirely by the authors

    Generative AI Use Disclosure The authors used generative AI tools solely for language editing and grammar polishing. All technical content, methodology, experimental design, and analysis were conducted entirely by the authors

  4. [12]

    Improving whispered speech recognition performance using pseudo-whispered based data augmentation,

    Z. Lin, T. Patel, and O. Scharenborg, “Improving whispered speech recognition performance using pseudo-whispered based data augmentation,” in2023 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), 2023, pp. 1–8

    2023

  5. [13]

    Aishell6-whisper: A chinese mandarin audio-visual whisper speech dataset with speech recognition baselines,

    C. Li, F. Su, J. Liu, H. Bu, Y . Wan, H. Suo, and M. Li, “Aishell6-whisper: A chinese mandarin audio-visual whisper speech dataset with speech recognition baselines,”arXiv preprint arXiv:2403.19833, 2024

    2024 arXiv

  6. [14]

    Whisper augmented end- to-end/hybrid speech recognition system—cyclegan approac,

    P. R. Gudepu, G. P. Vadisetti, A. Niranjan, K. Saranu, R. Sarma, M. A. B. Shaik, and P. Paramasivam, “Whisper augmented end- to-end/hybrid speech recognition system—cyclegan approac,” in Proc. Interspeech 2020, 2020, pp. 2302–2306

    2020

  7. [15]

    Learning representations by back-propagating errors,

    D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,”nature, vol. 323, no. 6088, pp. 533–536, 1986

    1986

  8. [16]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.”Iclr, vol. 1, no. 2, p. 3, 2022

    2022

  9. [17]

    backbone. •Uncertainty Perception Module (UPM):Our UPM is com- posed of a shared feature extractor (two 1D-CNN layers with GeLU and a single Transformer[18] encoder layer) followed by two task-specific heads for F0 prediction and masked spectrum reconstruction. •Training Detai...

  10. [18]

    Qwen3-asr technical report,

    X. Shi, X. Wang, Z. Guo, Y . Wang, P. Zhang, X. Zhang, Z. Guo, H. Hao, Y . Xi, B. Yanget al., “Qwen3-asr technical report,”arXiv preprint arXiv:2601.21337, 2025

    2025 arXiv

  11. [19]

    Fun-asr technical report,

    K. An, Y . Chen, Z. Chen, C. Deng, Z. Du, C. Gao, Z. Gao, B. Gong, X. Li, Y . Liet al., “Fun-asr technical report,”arXiv preprint arXiv:2509.12508, 2025

    2025

  12. [20]

    Wenetspeech: A 10000+ hours multi- domain mandarin corpus for speech recognition,

    B. Zhang, H. Lv, P. Guo, Q. Shao, C. Yang, L. Xie, X. Xu, H. Bu, X. Chen, C. Zenget al., “Wenetspeech: A 10000+ hours multi- domain mandarin corpus for speech recognition,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IE...

    2022

  13. [21]

    Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,

    G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhanget al., “Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,” arXiv preprint arXiv:2106.06909, 2021

    2021 arXiv

  14. [22]

    Aishell-1: An open- source mandarin speech corpus and a speech recognition base- line,

    H. Bu, J. Du, X. Na, B. Wu, and H. Zheng, “Aishell-1: An open- source mandarin speech corpus and a speech recognition base- line,” in2017 20th conference of the oriental chapter of the inter- national coordinating committee on speech databases and speech I/O systems and assess...

    2017

  15. [23]

    Lib- rispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an asr corpus based on public domain audio books,” in2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210

    2015

  16. [24]

    B. P. Lim,Computational differences between whispered and non- whispered speech. University of Illinois at Urbana-Champaign, 2011

    2011

  17. [25]

    Aishell6-whisper: A chinese mandarin audio-visual whisper speech dataset with speech recognition baselines,

    C. Li, F. Su, J. Liu, H. Bu, Y . Wan, H. Suo, and M. Li, “Aishell6-whisper: A chinese mandarin audio-visual whisper speech dataset with speech recognition baselines,”arXiv preprint arXiv:2509.23833, 2025

    2025

  18. [26]

    Computational differences between whispered and non-whispered speech,

    B. P. Lim, “Computational differences between whispered and non-whispered speech,” Ph.D. dissertation, University of Illinois at Urbana-Champaign, 2011

    2011

  19. [27]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

    2024 arXiv

  20. [28]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huanget al., “Qwen technical report,”arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  21. [29]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

    2017

  22. [30]

    Decoupled weight decay regulariza- tion,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regulariza- tion,”arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  23. [31]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.