Pith. sign in

REVIEW 4 major objections 5 minor 38 references

Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that, in visual speech recognition, larger language-model decoders are consistently more accurate, and that giving the LLM the preceding textual context plus letting it iteratively refine its own transcription yields a…

desk verdict Useful empirical scaling result for LLM-based VSR, but the context-aware gains are measured with ASR-derived audio context that a silent-video VSR cannot rely on. read the letter →

arxiv 2506.02012 v1 pith:EKRPPOQA submitted 2025-05-27 cs.CV cs.SDeess.AS

Zehua Liu , Xiaolou Li , Li Guo , Lantian Li , Dong Wang This is my paper
classification cs.CVcs.SDeess.AS
keywords visualspeechrecognitionlargelanguagemodelsscalinglawscontext-awaredecodingiterativepolishinglipreadingCNVSRC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that large language models, used as decoders for visual speech recognition, obey a scaling law: larger LLMs produce lower character error rates. It also introduces two decoding strategies, giving the LLM the transcript of the preceding 30 seconds of audio as prompt context and letting it iteratively refine its own output, and reports that combining them on top of Qwen2.5-32B-Instruct lowers CER on the CNVSRC.Single test set from 49.32% (Transformer decoder) to 38.18%. If true, this means LLM-based decoding with context and self-correction is a practical path to much better lip-reading systems, especially for Chinese continuous VSR.

What carries the argument

The system is a VSR-LLM: a visual encoder produces lip-movement features, a linear connector projects them into the LLM's embedding space, and a fine-tuned LLM generates the transcription. The three interventions are (1) swapping the Transformer decoder for larger LLMs (Qwen2.5-7B/14B/32B/32B-Instruct), (2) context-aware decoding, which prepends the ASR-transcribed text of the preceding 30-second segment to the prompt, and (3) iterative polishing, where the model's previous decoding is fed back as a 'reference' prompt so the model self-corrects over up to four passes. The loss is cross-entropy with QLoRA fine-tuning; the combined method trains on context plus previous decoding and uses a maximum of four iterations at inference.

What would settle it

Run the context-aware iterative decoding model on the CNVSRC.Single test set with the preceding-text prompt removed or replaced by a random text of matched length (identical vocabulary and length) and measure CER. If the CER improvement over the no-context baseline disappears or shrinks substantially, the reported benefit depends on audio-derived context rather than on the model's ability to exploit arbitrary context text.

Watch

Extended reading notes

Core claim

The central claim is that LLMs can be harnessed more fully in VSR than earlier work did, and that three mechanisms account for the gains: scaling the LLM decoder, feeding it the preceding textual context during decoding, and asking it to polish its own transcription over several iterations. On the CNVSRC.Single test set, each mechanism improves CER, the gains combine additively, and the final context-aware iterative decoding system with Qwen2.5-32B-Instruct reaches 38.18% CER versus 49.32% for the Transformer decoder baseline and 41.86% for the same LLM without context or iteration.

Load-bearing premise

The context-aware gains assume that an accurate text transcript of the 30 seconds of audio before each video clip is available at test time, but a genuinely silent-video lip-reading application would have no such audio.

Editorial extensions

If this is right

  • Larger LLM decoders (beyond 32B parameters) should keep improving VSR accuracy, provided the architecture and training budget are held fixed.
  • Supplying any sufficiently topical preceding text—not necessarily ASR output—should give similar context gains, since the paper reports little difference across context types.
  • Iterative polishing will give a further, though smaller, improvement on top of context-aware decoding, and the gains should grow if the model is fine-tuned specifically on the refinement task.
  • The same VSR-LLM recipe should transfer to related tasks such as visual speech synthesis, as the paper suggests in its outlook.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the scaling law holds beyond 32B, the practical limit on VSR quality may move from the visual encoder to the language model's world knowledge, making model size a primary axis for lip-reading progress.
  • The context assumption could be relaxed by training the model to accept video-only or subtitle-derived context, which would make the method applicable to truly silent surveillance and hearing-assist devices; that is a direct, testable extension of the paper's design.
  • A controlled comparison of ASR-based, subtitle-based, and keyword-based contexts would test the footnote's claim that context type does not matter and would identify the cheapest usable context source for deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents VSR-LLM, a visual speech recognition system that uses a pre-trained visual encoder, a Qwen2.5 LLM decoder fine-tuned with QLoRA, and three proposed components: a scaling test across LLM sizes, context-aware decoding (CAD) that prepends the preceding 30-second ASR transcription to the prompt, and iterative polishing (IP) that refines the output over multiple decoding passes. These are combined into context-aware iterative decoding (CAID). On the CNVSRC.Single Mandarin VSR dataset, the authors report that larger LLMs yield lower CER, CAD reduces CER by 2.9-3.7 points with the 32B-Instruct model, IP gives smaller gains, and CAID yields the best result of 38.18% CER on the test set, versus 41.86% for the 32B-Instruct baseline without context and 49.32% for a Transformer decoder.

Significance. If the results hold, the paper provides useful empirical evidence that LLM decoders scale in VSR, that in-context text can help disambiguate homophones and domain-specific terms, and that iterative self-correction offers modest gains. The work is clearly presented, and the decision to release code, datasets, and results is commendable and helps reproducibility. However, the central CAD/CAID claims depend on an inference-time context that is not available in the silent-video VSR setup defined in Section 1, and the quantitative support for IP is statistically weak because all results are single runs without variance reporting. The scaling-law claim is based on only four checkpoints from one LLM family, one of which is an instruction-tuned variant, so the phrase 'confirming a scaling law' is stronger than the evidence supports.

major comments (4)
  1. [Sections 2.4, 3.1, and footnote 1] The context T used for CAD and CAID is the ASR transcription of the 30-second audio segment preceding each clip, as stated in Section 3.1. This makes the reported gains in Tables 2 and 4 (e.g., 41.86% to 38.18% for CAID) contingent on access to audio, which is not part of the silent-video VSR task defined in Section 1. Footnote 1 asserts that 'there is not much difference for various types of contexts' but no ablation is reported with video-only, subtitle, keyword, or previously-recognized VSR text as T. As written, the paper does not establish that context-aware decoding improves VSR in the setting that the paper itself defines. The authors should either provide such an ablation or explicitly scope the contribution to audio-visual speech recognition where ASR context is available.
  2. [Tables 2-4 and Section 4.3] Every reported CER number is from a single run, with no error bars, confidence intervals, or statistical significance tests. This is particularly problematic for Iterative Polishing, where the gains in Table 3 are only 0.40-0.68 CER points (e.g., 41.86% to 41.35% for the test set with 32B-Instruct). Given typical run-to-run variance when fine-tuning LLMs with QLoRA, these differences are within plausible noise. The claim that 'Iterative Polishing progressively reduces recognition errors' is not yet supported. The authors should report multiple seeds or at least provide variance estimates and, ideally, a paired significance test for the IP and CAID comparisons.
  3. [Section 4.1 and Table 1] The scaling-law conclusion rests on four checkpoints: Qwen2.5-7B, 14B, 32B, and 32B-Instruct. The Instruct version differs not only in parameter count but also in instruction tuning, so the monotonic decrease from 45.20% to 41.86% conflates model scale with alignment. Furthermore, the paper does not fit the scaling trend to a parametric law or report any uncertainty/confidence intervals. Therefore, the statement that the results 'confirm a scaling law in the VSR task' is an overclaim. I suggest rephrasing to 'larger LLMs in the Qwen2.5 family yield better CER' and, if the scaling-law claim is to remain, adding a proper fit with error bars across multiple seeds.
  4. [Equations (3)-(4) and Section 2.4] The training objectives for IP and CAID sum two cross-entropy terms: one conditioned on the previous decoding result P and one not. The paper does not specify how P is generated during training (e.g., a frozen baseline model versus the current model in training), how often P is updated, or whether the two terms are weighted equally. The prompts in Section 2.4 show 'reference: {None}' for the first iteration and 'reference: {Previous decoding result }' for later iterations, but the loss in Eq. (4) conditions on both P and T simultaneously, which does not match the shown first-iteration prompt (which has no previous result). This mismatch between the training objective and the inference procedure undermines reproducibility. The authors should clarify the exact training protocol, including how P is sampled and how the loss terms are combined.
minor comments (5)
  1. [Section 2.3] There is a typo: 'aproach' should be 'approach'.
  2. [Tables 1-4] The table headers show 'V alid' with a space; please fix the column header to 'Valid'.
  3. [Section 3.4] The decoding hyperparameters include a repetition penalty of 6, which is unusually high; please clarify the range and how it was chosen, as this hyperparameter can have a large effect on generated text quality.
  4. [Section 2.4] The prompts are presented as raw text but are not numbered consistently with the 'Prompt 1-4' references in Figure 1; adding labels such as 'Prompt 5' and 'Prompt 6' would improve readability.
  5. [Abstract] The phrase 'by these designs' is awkward; consider revising to 'with these designs' or 'using these methods'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's headline results are empirical comparisons, and the ASR-derived context issue is an applicability assumption rather than a logical circularity.

full rationale

The paper makes no formal derivation whose conclusion is forced by its premises. The scaling-law claim is an observed monotonic CER trend across Qwen2.5-7B/14B/32B/32B-Instruct in Table 1, with no parameter fitted and then relabeled as a prediction. Context-aware decoding in Eq. (2) conditions the LLM on preceding text T, which is obtained independently via ASR from the prior 30-second audio (Section 3.1); although this raises a real deployment concern for a strictly silent-video VSR system, T is not the target y, and the reported CER gains are measured on held-out test clips, so this is an input-conditioning assumption rather than a circular definition. Iterative polishing in Eq. (3) and CAID in Eq. (4) feed the model's own previous output P as auxiliary prompt text, but the loss is supervised against ground-truth y, so the output is not forced to equal the input; P is a conditioning variable, not the training target. Self-citations such as Refs. [10], [29], and [30] supply prior work, the dataset, and the ASR tool, but none is used as a uniqueness theorem or as the sole justification of the paper's central claims. The footnote asserting 'there is not much difference for various types of contexts' is unsupported and relevant to external validity, but it is not a derivation reducing to its own inputs. Therefore no circular step is present; the conclusions rest on empirical comparisons and are not forced by construction.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical study with no fitted theoretical parameters. The free parameters are hand-chosen hyperparameters. The load-bearing assumptions are the availability of ASR-derived context, the representativeness of CNVSRC.Single, and the sufficiency of four model sizes to establish a scaling trend. No new physical or theoretical entities are introduced.

free parameters (7)
  • beam width = 5
    Beam search decoding hyperparameter set in Section 3.4; no ablation is reported.
  • repetition penalty = 6
    Beam search repetition penalty set to 6 in Section 3.4; this is unusually high and no ablation is given.
  • inference iterations = 4
    Maximum iterations for iterative polishing in Section 3.4; no ablation or convergence curve is shown.
  • context window length = 30 seconds
    Preceding audio segment used to create context text, Section 3.1; no ablation on window length is reported.
  • training steps = 17000
    All models trained for 17,000 steps, Section 3.4; chosen without reported convergence analysis.
  • loss weighting in Eq. (3) and (4) = equal weights (1 and 1)
    The two cross-entropy terms in iterative and context-aware objectives are summed with equal weight; no lambda or ablation is reported.
  • QLoRA configuration = not reported
    Rank, alpha, target modules, learning rate, and batch size are omitted, though they directly affect fine-tuning results.
assumptions (4)
  • domain assumption Preceding 30-second textual context is available at inference and improves decoding.
    Section 2.4 footnote 1 states 'The preceding context is presumed to be available'; experiments only use ASR-derived text from audio, not available in truly silent video.
  • domain assumption CNVSRC.Single is an appropriate testbed and conclusions transfer to other VSR settings.
    Section 3.1 excludes LRS2/LRS3 because clips are short; the paper does not test generalization to short-clip or multi-domain benchmarks.
  • domain assumption Four Qwen2.5 model sizes are sufficient to establish a scaling law.
    Section 4.1 uses 7B, 14B, 32B, and 32B-Instruct; no fit, no error bars, and only one model family are used.
  • domain assumption Monotonic CER improvement is statistically meaningful without repeated runs.
    Tables 1-4 report single runs with no variance or significance tests, so small gains may be noise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing." pith.science (2026). https://pith.science/paper/EKRPPOQA

@misc{pith2026250602012,
  author       = {Pith},
  title        = {Pith review of: Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EKRPPOQA}},
  note         = {Machine review of arXiv:2506.02012}
}
read the original abstract

Visual Speech Recognition (VSR) transcribes speech by analyzing lip movements. Recently, Large Language Models (LLMs) have been integrated into VSR systems, leading to notable performance improvements. However, the potential of LLMs has not been extensively studied, and how to effectively utilize LLMs in VSR tasks remains unexplored. This paper systematically explores how to better leverage LLMs for VSR tasks and provides three key contributions: (1) Scaling Test: We study how the LLM size affects VSR performance, confirming a scaling law in the VSR task. (2) Context-Aware Decoding: We add contextual text to guide the LLM decoding, improving recognition accuracy. (3) Iterative Polishing: We propose iteratively refining LLM outputs, progressively reducing recognition errors. Extensive experiments demonstrate that by these designs, the great potential of LLMs can be largely harnessed, leading to significant VSR performance improvement.

Figures

Figures reproduced from arXiv: 2506.02012 by the authors.

Figure 1
Figure 1. Our VSR-LLM architecture. shortly. The input text prompt is tokenized using a LLM To￾kenizer and mapped into an embedding space, resulting in the text embedding. To effectively handle multi-modal inputs, we adopt a strategy inspired by Qwen2-VL [23] that concatenates the text and visual embedding vectors. Three special tokens are introduced to perform the concatenation: • <|endoftext|> marks the end of the textual i… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 24 canonical work pages

  1. [1]

    Typically, a VSR system takes a silent video containing the speaker’s lip movements as input and outputs the corresponding text

    Introduction Visual Speech Recognition (VSR), often referred to as lip read- ing, is a technique that transcribes spoken content by analyzing a speaker’s lip movements [1, 2, 3]. Typically, a VSR system takes a silent video containing the speaker’s lip movements as input and outputs the corresponding text. VSR has a wide range of real-world applications, ...

    work page

  2. [2]

    Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing

    Our VSR-LLM Architecture 2.1. Main Architecture The architecture of our VSR-LLM system is adapted from [19], and has been illustrated in Figure 1. The input is a preprocessed silent video containing the mouth area of the speaker’s face. A Visual Encoder extracts frame-wise visual features from the in- put video. The extracted visual features are then pass...

    work page Pith review arXiv 2025

  3. [3]

    Data To evaluate the effectiveness of the proposed methods, we need to select an appropriate dataset that enables LLMs to demon- strate their potential

    Experiment Setup 3.1. Data To evaluate the effectiveness of the proposed methods, we need to select an appropriate dataset that enables LLMs to demon- strate their potential. Popular VSR datasets like LRS2 and LRS3 [28] contain relatively short video clips (usually under 10 seconds), limiting the availability of contextual information and thus not suitabl...

    work page 2023

  4. [4]

    Results on Scaling Test We first conduct a Scaling Test to investigate how the LLM scale affects VSR performance

    Experimental Results 4.1. Results on Scaling Test We first conduct a Scaling Test to investigate how the LLM scale affects VSR performance. The input prompt is in the form of Prompt 1 in Figure 1. The experimental results are presented in Table 1. Firstly, we can see that LLMs significantly outperform the Transformer decoder baseline, highlighting the adv...

    work page

  5. [5]

    By a comprehensive experimental study, several interesting conclu- sions can be drawn

    Conclusion In this paper, we explored the integration of Large Language Models (LLMs) into Visual Speech Recognition (VSR). By a comprehensive experimental study, several interesting conclu- sions can be drawn. First, we confirmed the existence of the LLM’s scaling law in VSR tasks, demonstrating that larger LLMs consistently yield better recognition perf...

    work page

  6. [6]

    LipNet: Sentence-level lipreading,

    Y . M. Assael, B. Shillingford, S. Whiteson, and N. De Fre- itas, “LipNet: Sentence-level lipreading,” arXiv preprint arXiv:1611.01599, 2016

    arXiv 2016

  7. [7]

    Large-scale visual speech recognition,

    B. Shillingford, Y . Assael, M. W. Hoffman, T. Paine, C. Hughes, U. Prabhu, H. Liao, H. Sak, K. Rao, L. Bennett, M. Mulville, M. Denil, B. Coppin, B. Laurie, A. Senior, and N. de Fre- itas, “Large-scale visual speech recognition,” in INTERSPEECH, 2019, pp. 4135–4139

    work page 2019

  8. [8]

    Auto-A VSR: Audio-visual speech recognition with automatic labels,

    P. Ma, A. Haliassos, A. Fernandez-Lopez, H. Chen, S. Petridis, and M. Pantic, “Auto-A VSR: Audio-visual speech recognition with automatic labels,” in ICASSP 2023-2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    work page 2023

Show all 38 references
  1. [9]

    Zero- shot fake video detection by audio-visual consistency,

    X. Li, Z. Liu, C. Chen, L. Li, L. Guo, and D. Wang, “Zero- shot fake video detection by audio-visual consistency,” inINTER- SPEECH, 2024, pp. 2935–2939

    2024

  2. [10]

    End-to-end audio-visual speech recognition with conformers,

    P. Ma, S. Petridis, and M. Pantic, “End-to-end audio-visual speech recognition with conformers,” in ICASSP 2021-2021 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 7613–7617

    2021

  3. [11]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution-augmented transformer for speech recognition,” in INTERSPEECH, 2020, pp. 5036–5040

    2020

  4. [12]

    Audio-visual speech recognition with a hybrid CTC/attention architecture,

    S. Petridis, T. Stafylakis, P. Ma, G. Tzimiropoulos, and M. Pan- tic, “Audio-visual speech recognition with a hybrid CTC/attention architecture,” in 2018 IEEE Spoken Language Technology Work- shop (SLT). IEEE, 2018, pp. 513–520

    2018

  5. [13]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems , 2017

    2017

  6. [14]

    SyncVSR: Data-efficient visual speech recognition with end-to-end cross- modal audio token synchronization,

    Y . J. Ahn, J. Park, S. Park, J. Choi, and K.-E. Kim, “SyncVSR: Data-efficient visual speech recognition with end-to-end cross- modal audio token synchronization,” in INTERSPEECH, 2024, pp. 867–871

    2024

  7. [15]

    AlignVSR: Audio-visual cross-modal alignment for visual speech recogni- tion,

    Z. Liu, X. Li, C. Chen, L. Guo, L. Li, and D. Wang, “AlignVSR: Audio-visual cross-modal alignment for visual speech recogni- tion,” arXiv preprint arXiv:2410.16438, 2024

    2024 arXiv

  8. [16]

    Video understanding with large language models: A survey,

    Y . Tang, J. Bi, S. Xu, L. Song, S. Liang, T. Wang, D. Zhang, J. An, J. Lin, R. Zhu et al. , “Video understanding with large language models: A survey,” arXiv preprint arXiv:2312.17432, 2023

    2023

  9. [17]

    Audio-visual LLM for video understanding,

    F. Shu, L. Zhang, H. Jiang, and C. Xie, “Audio-visual LLM for video understanding,” arXiv preprint arXiv:2312.06720, 2023

    2023 arXiv

  10. [18]

    A survey on generative AI and LLM for video generation, understanding, and streaming,

    P. Zhou, L. Wang, Z. Liu, Y . Hao, P. Hui, S. Tarkoma, and J. Kangasharju, “A survey on generative AI and LLM for video generation, understanding, and streaming,” arXiv preprint arXiv:2404.16038, 2024

    2024 arXiv

  11. [19]

    In- teractive video search with multi-modal LLM video captioning,

    Y .-T. Cheng, J. Wu, Z. Ma, J. He, X.-Y . Wei, and C.-W. Ngo, “In- teractive video search with multi-modal LLM video captioning,” in International Conference on Multimedia Modeling . Springer, 2025, pp. 302–309

    2025

  12. [20]

    Improv- ing image captioning descriptiveness by ranking and LLM-based fusion,

    S. Bianco, L. Celona, M. Donzella, and P. Napoletano, “Improv- ing image captioning descriptiveness by ranking and LLM-based fusion,” arXiv preprint arXiv:2306.11593, 2023

    2023

  13. [21]

    From Alt-text to real context: Revolutionizing image captioning using the potential of LLM,

    V . Patel, A. Modi, H. Mistry, A. Mishra, R. Upadhyay, and A. Shah, “From Alt-text to real context: Revolutionizing image captioning using the potential of LLM,”IJS-CSEIT, vol. 11, no. 1, pp. 379–387, 2025

    2025

  14. [22]

    Image captioning using multimodal LLMs,

    N. Tiet, “Image captioning using multimodal LLMs,” LU-CS-EX, 2025

    2025

  15. [23]

    Large language models are strong audio-visual speech recognition learners,

    U. Cappellazzo, M. Kim, H. Chen, P. Ma, S. Petridis, D. Falav- igna, A. Brutti, and M. Pantic, “Large language models are strong audio-visual speech recognition learners,” arXiv preprint arXiv:2409.12319, 2024

    2024 arXiv

  16. [24]

    Where visual speech meets language: VSP-LLM framework for efficient and context- aware visual speech processing,

    J. H. Yeo, S. Han, M. Kim, and Y . M. Ro, “Where visual speech meets language: VSP-LLM framework for efficient and context- aware visual speech processing,” in EMNLP, 2024, pp. 11 391– 11 406

    2024

  17. [25]

    Learning audio-visual speech representation by masked multimodal cluster prediction,

    B. Shi, W.-N. Hsu, K. Lakhotia, and A. Mohamed, “Learning audio-visual speech representation by masked multimodal cluster prediction,” in International Conference on Learning Representa- tions, 2022

    2022

  18. [26]

    QLoRA: efficient finetuning of quantized LLMs,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: efficient finetuning of quantized LLMs,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  19. [27]

    The Llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Let- man, A. Mathur, A. Schelten, A. Yang, A. Fan et al., “The Llama 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  20. [28]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge et al. , “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,”arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  21. [29]

    Seed-ASR: Understanding diverse speech and contexts with LLM-based speech recognition,

    Y . Bai, J. Chen, J. Chen, W. Chen, Z. Chen, C. Ding, L. Dong, Q. Dong, Y . Du, K. Gaoet al., “Seed-ASR: Understanding diverse speech and contexts with LLM-based speech recognition,” arXiv preprint arXiv:2407.04675, 2024

    2024 arXiv

  22. [30]

    SALM: Speech- augmented language model with in-context learning for speech recognition and translation,

    Z. Chen, H. Huang, A. Andrusenko, O. Hrinchuk, K. C. Puvvada, J. Li, S. Ghosh, J. Balam, and B. Ginsburg, “SALM: Speech- augmented language model with in-context learning for speech recognition and translation,” in ICASSP 2024-2024 IEEE Inter- national Conference on Acoustics,...

    2024

  23. [31]

    End-to-end speech recognition contextualization with large language models,

    E. Lakomkin, C. Wu, Y . Fathullah, O. Kalinli, M. L. Seltzer, and C. Fuegen, “End-to-end speech recognition contextualization with large language models,” in ICASSP 2024-2024 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 1...

    2024

  24. [32]

    Eliciting the priors of large lan- guage models using iterated in-context learning,

    J.-Q. Zhu and T. L. Griffiths, “Eliciting the priors of large lan- guage models using iterated in-context learning,” arXiv preprint arXiv:2406.01860, 2024

    2024 arXiv

  25. [33]

    Deep audio-visual speech recognition,

    T. Afouras, J. S. Chung, A. Senior, O. Vinyals, and A. Zisser- man, “Deep audio-visual speech recognition,” IEEE transactions on pattern analysis and machine intelligence , vol. 44, no. 12, pp. 8717–8727, 2018

    2018

  26. [34]

    CNVSRC 2023: The first Chinese continuous visual speech recognition challenge,

    C. Chen, Z. Liu, X. Li, L. Li, and D. Wang, “CNVSRC 2023: The first Chinese continuous visual speech recognition challenge,” in INTERSPEECH, 2024, pp. 1930–1934

    2023

  27. [35]

    CN-CVS: A Mandarin audio-visual dataset for large vocabulary continuous visual to speech synthesis,

    C. Chen, D. Wang, and T. F. Zheng, “CN-CVS: A Mandarin audio-visual dataset for large vocabulary continuous visual to speech synthesis,” inICASSP 2023-2023 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  28. [36]

    Reti- naface: Single-shot multi-level face localisation in the wild,

    J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, “Reti- naface: Single-shot multi-level face localisation in the wild,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 5203–5212

    2020

  29. [37]

    How far are we from solving the 2D & 3D face alignment problem?(and a dataset of 230,000 3D facial landmarks),

    A. Bulat and G. Tzimiropoulos, “How far are we from solving the 2D & 3D face alignment problem?(and a dataset of 230,000 3D facial landmarks),” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 1021–1030

    2017

  30. [38]

    Qwen2. 5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Weiet al., “Qwen2. 5 technical report,”arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.