Pith. sign in

REVIEW 3 major objections 7 minor 5 cited by

LLaVA-SLT: Visual Language Tuning for Sign Language Translation

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A three-stage multimodal recipe brings gloss-free sign language translation to the level of gloss-supervised models.

desk verdict A technically solid gloss-free SLT pipeline with honest ablations, but the headline 'closes to gloss-based' rests on an unreleased scripted dataset whose gain cannot be separated from test-domain overlap. read the letter →

arxiv 2412.16524 v1 pith:FQ4WLK7P submitted 2024-12-21 cs.CV

Han Liang , Chengyu Huang , Yuecheng Xu , Cheng Tang , Weicai Ye , Juze Zhang , Xin Chen , Jingyi Yu
show 1 more author
Lan Xu
This is my paper · ORCID
classification cs.CV
keywords signlanguagetranslationgloss-freelargemultimodalmodelvisualcontrastivepretraininghierarchicalencoderCSL-400hLLMscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sign language translation has long depended on gloss annotations—word-by-word labels of signs—that are costly and slow to produce. The paper argues that a large multimodal model, adapted to sign language with a three-stage training scheme, can translate from raw video without glosses, and that adding annotation-free video data closes most of the gap with gloss-based systems. On the CSL-Daily benchmark, the model reaches a BLEU-4 of 20.42 with the standard training set, beating prior gloss-free approaches, and 25.23 when an additional 401 hours of unlabeled-style video is added, close to the 25.79 of a strong gloss-supervised baseline. The result matters because it points to a scalable path for sign language translation: collecting raw video with text descriptions rather than paying experts to gloss it.

What carries the argument

The load-bearing mechanism is the hierarchical visual encoder: a frozen DINOv2 backbone (finetuned with LoRA) provides frame-level features; a local-attention transformer pools them into isolated word-level tokens; and a full-attention transformer encodes the token sequence into a sentence-level query. This visual "language" is aligned to a pretrained multilingual text encoder (mBART) via an outer CLIP-style contrastive loss plus an inner SignCL loss, and then mapped into the language-model token embedding space by a two-layer MLP connector while the language model and visual encoder are frozen. The response prompting—a task prompt and a format prompt—constrains the output space of the language model, which the ablations show is important for translation accuracy.

What would settle it

Train LLaVA-SLT on CSL-400h and evaluate on a sign language test set recorded in natural, uncontrolled conditions (for example, conversational or broadcast sign video with human-written references); if the BLEU-4 advantage over the lab-only baseline shrinks substantially, the gap-closing result is an artifact of matching lab conditions. Alternatively, an external audit checking CSL-400h for near-duplicate videos or lexical overlap with the CSL-Daily test set could settle whether the extra-data gain is genuine.

Watch

Extended reading notes

Core claim

The central claim is that a three-stage "trilogy"—continued pretraining of a scaled-up language model on sign-language textual corpora, contrastive pretraining of a hierarchical visual encoder against a frozen text encoder, and end-to-end visual language tuning with a lightweight MLP connector—produces word-level visual tokens that behave like a "visual language" compatible with language-model token embeddings. The paper demonstrates that this design, LLaVA-SLT, outperforms previous gloss-free methods on CSL-Daily and Phoenix-2014T, and that when training data is expanded with the annotation-free CSL-400h dataset, its BLEU-4 approaches the gloss-based TwoStream-SLT model. The authors interpret this as evidence that gloss-free translation can be scaled with annotation-free multimodal data, potentially replacing labor-intensive glossing.

Load-bearing premise

The paper's headline claim that annotation-free data closes the gap with gloss-based models rests on the newly collected CSL-400h dataset—264,461 videos recorded in a controlled lab with text generated by a large language model—which is assumed to be a representative, unbiased sample of Chinese Sign Language; the dataset is not publicly released and its collection protocol is not independently audited.

Editorial extensions

If this is right

  • If the result holds, sign language translation research can shift from expensive gloss annotation to collecting raw video paired with text, which is available at web scale.
  • Scaling the base language model from 3B to 14B and scaling video data from 20 to 400 hours both improve BLEU scores, suggesting a predictable scaling path for the approach.
  • Word-level visual tokens outperform sentence-level features as inputs to the language model, giving a concrete design rule for future sign-language multimodal models.
  • The model transfers to German sign language without continued linguistic pretraining, so the visual component of the recipe is language-agnostic.
  • Higher input resolution (336 squared) and a larger visual backbone materially improve accuracy, implying that fine-grained hand and face cues are the dominant signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the extra-data gains come from the distributional similarity of CSL-400h and CSL-Daily—both recorded in controlled laboratories—the benefit may shrink on real-world footage; an independent natural test set would settle this.
  • The text side of CSL-400h was generated by a large language model, so the dataset may encode a canonical, well-formed style of Chinese that inflates BLEU scores against textbook references; training on naturally occurring sign language text could yield lower but more realistic numbers.
  • A direct test of the framework's scaling claim would be to apply it to open-domain sign video with automatic captions and measure whether BLEU continues to rise with hundreds of additional hours of unlabeled data.
  • The reliance on a frozen visual backbone (DINOv2) means the "visual language" is built on features not optimized for sign; an end-to-end pretraining of the whole encoder might remove that constraint, though this is not explored in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces LLaVA-SLT, a large multimodal model framework for gloss-free sign language translation. The method is trained in three stages: (1) linguistic continued pretraining, in which a Qwen-2.5 LLM is further trained on a newly collected CSL-Corpus of sign-language-related text; (2) visual contrastive pretraining, in which a hierarchical DINOv2-based visual encoder with frame-, word-, and sentence-level features is aligned to an mBART text encoder using CLIP and SignCL losses; and (3) visual language tuning, in which a lightweight MLP connector maps word-level visual tokens into the LLM embedding space while the visual encoder and LLM are frozen, optionally followed by full tuning with LoRA. The model is evaluated on CSL-Daily and Phoenix-2014T. The paper reports state-of-the-art gloss-free results on both benchmarks and, when adding a newly collected 401-hour annotation-free dataset CSL-400h, claims that the model closely approaches gloss-based translation accuracy on CSL-Daily.

Significance. If the results hold, the paper makes a concrete contribution: it demonstrates that a scaled LLM with a visually pretrained hierarchical encoder can reduce the gap between gloss-free and gloss-based sign language translation, and it provides a systematic ablation of LLM scale, visual encoder architecture, input resolution, data scaling, connector design, and prompting. The ablations are unusually thorough for this problem and give useful evidence about which components matter. The main limitation, as discussed below, is that the headline 'closes to gloss-based accuracy' result depends on an unreleased, internally constructed dataset whose distributional relationship to the test set is not analyzed; additionally, there is an unresolved numerical inconsistency in the reported BLEU-4 scores for the same model. The paper does not provide code or data links despite a statement in the broader-impact section about open-sourcing, so the central extra-data claim is not currently reproducible.

major comments (3)
  1. [Table 3 and Table 6] There is an internal inconsistency in the reported BLEU-4 for the same model. Table 3 lists LLaVA-SLT (ours) with BLEU-1 52.15, BLEU-4 20.42, and ROUGE 51.26, while Table 6 row 4 reports the same configuration (Qwen-2.5 14B after continued pretraining) with BLEU-1 52.15, BLEU-4 20.92, and ROUGE 51.26. Because the BLEU-1 and ROUGE values are identical, these rows evidently refer to the same model, so the BLEU-4 discrepancy is either a typo or a difference in evaluation protocol. This must be resolved, since the headline comparison against GFSLT-VLP-SignCL (16.16 vs. 20.42) and the LLM-scaling trend in Table 6 depend on which number is correct.
  2. [Table 3 and Supplement Sec. B] The abstract's claim that the method 'closes to the gloss-based accuracy' rests on the last row of Table 3, which adds CSL-400h data and raises BLEU-4 from 20.42 to 25.23. However, CSL-400h is not released, and its construction, described in Supplement Sec. B, involves GPT-4-generated text prompts from predefined scenario categories (work, campus, shopping, family, entertainment, medical, others) that are the same broad categories as in CSL-Daily, with videos recorded in a controlled lab setting. The paper reports no overlap analysis between CSL-400h prompts and CSL-Daily test references, no held-out signer or scenario evaluation, and no baseline trained on the same added data. As presented, the marginal gain cannot be attributed to the method's scalability rather than to distributional alignment between the extra training data and the test set. To support the claim, the authors should either release the dataset, or provide experiments that control for domain overlap, such as a signer- and scenario-split evaluation, a comparison baseline using the same extra data with an existing method, or an analysis of prompt/test overlap. Without such evidence, the 'closes to gloss-based accuracy' claim should be substantially softened or removed.
  3. [Table 7 and Section 4.4] The data-scaling ablation in Table 7 rows 4-5 is not fully specified. The text says the training data is expanded from CSL-Daily to 200 and 400 hours using CSL-400h, but it does not state whether these models also use CSL-400h during visual language tuning, how the '50% extra data' subset is sampled, or how the extra-data rows in Table 7 relate to the 'w/ extra data' row in Table 3. This ambiguity matters because the final translation metric aggregates the effects of visual pretraining and tuning. Please state the exact training protocol for each configuration.
minor comments (7)
  1. [Abstract and Section 4.2] The phrase 'closes to the gloss-based accuracy' is grammatically awkward and imprecise; I suggest 'closes the gap to gloss-based methods' or 'approaches gloss-based accuracy'.
  2. [Table 3 caption] The caption says 'with the same training set (ours)' but this is ambiguous; please specify explicitly that 'ours' in the row without extra data is trained only on the CSL-Daily training split.
  3. [References] Reference [48] lists 'LoRA: Low-rank adaptation of large language models' with an incomplete year '202.'; please correct it.
  4. [Section 3.2] The relationship between the local attention window size and the nearest-neighbor downsampling step (step 4) is not explained; please define how a 'word' is determined for the word-level visual tokens.
  5. [Figure 1 caption] The phrase 'tamed for sign language translation' should be 'tuned for sign language translation' to match the terminology used in the rest of the paper.
  6. [Section 4.2] The sentence 'it even closes to the gloss-based accuracy' appears in prose as well; also 'surpasses gloss-based methods in BLEU-1 scores' should explicitly say 'BLEU-1 score on the CSL-Daily test set' for precision.
  7. [Supplement Sec. D] The limitation about 'Real-world Scenarios' is stated only in the supplement; consider adding a sentence in the main conclusion to acknowledge that the reported results are on lab-recorded data.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: all reported results are genuine held-out predictions; the CSL-400h data concern is a generalization/verifiability issue, not a circular one.

full rationale

The paper's derivation chain is self-contained against external benchmarks rather than self-referential. Linguistic continued pretraining (Sec. 3.1) trains the LLM on CSL-Corpus, a separately collected text corpus; visual contrastive pretraining (Sec. 3.2) aligns a DINOv2-based hierarchical encoder with the external mBART text encoder using CLIP and SignCL losses; visual language tuning (Sec. 3.3) freezes these pretrained models and trains only an MLP connector. Final numbers in Tables 3 and 4 are evaluated on the held-out CSL-Daily and Phoenix-2014T test splits, so no reported BLEU/ROUGE value is a fitted parameter or a renamed training objective. The 'closes to gloss-based' claim rests on training with the newly collected CSL-400h dataset, but the evaluation is still on the external CSL-Daily test set, so the improvement is an empirical prediction rather than a construction. The skeptical concern that CSL-400h is GPT-4-scripted, unreleased, and scenario-aligned with CSL-Daily is a legitimate external-validity and contamination risk, and the paper itself acknowledges in Supplement Sec. D that all training and evaluation datasets are captured in controlled settings and may not represent natural environments; however, this is not circularity as defined here, because no equation or fitted value in the paper reduces to the test target. The only self-citations (e.g., OMG [68] and LL3DA [22] as examples of cross-modal LMMs) appear in related-work background and do not carry any load-bearing argument of the SLT method. Accordingly, no circular step is exhibited and the score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 3 invented entities

The central results rest on two new, unreleased resources: the CSL-400h video-text dataset and the CSL-Corpus text corpus, whose text portions were partly generated by GPT-4. All other components are standard pretrained models (DINOv2, mBART, Qwen, LLaMA) and standard losses (CLIP, SignCL, cross-entropy). The hyperparameters listed are design choices rather than fitted to the benchmark, but several (SignCL weight, local attention window) are not justified by ablations. The hierarchy assumption, that local video windows correspond to sign-language words, is a heuristic not supported by gloss annotations.

free parameters (4)
  • SignCL weight lambda = 1e-2
    Balances inner and outer contrastive losses in Eq. (2); set by hand, no ablation is shown.
  • Local attention downsample step = step 4
    Nearest-neighbor downsample applied to word-level features; design choice for word segmentation not independently justified.
  • LoRA rank and alpha = rank 8, alpha 16
    Used for DINOv2 and LLM tuning; common default values, not ablated.
  • Prompt template constraints = single sentence, task and format prompts
    Response prompting narrows the output space; ablations show sensitivity, so the prompt is a hand-chosen element of the system.
assumptions (5)
  • domain assumption DINOv2 and mBART pretrained representations transfer to sign language video and text.
    Visual and text encoders are used with frozen pretrained weights or LoRA fine-tuning (Sec. 3.2); no evidence beyond benchmark results that they capture sign language phonology.
  • domain assumption Sign language videos can be decomposed into isolated word-level local motions.
    The hierarchical encoder assumes local attention windows correspond to sign words (Sec. 3.2, Figure 4); no gloss alignment is used to verify this.
  • domain assumption GPT-4 generated text and scraped webpages are a representative sample of sign language linguistic structure.
    CSL-Corpus gloss-text pairs use GPT-4 to generate spoken text and dictionary glosses (Supplement B); this may not reflect real signer usage.
  • domain assumption Continued pretraining on sign-related text improves translation from visual input.
    Ablations in Table 6 show gains, but only for Chinese; the paper skips this stage for German and still obtains SOTA, so its necessity is language-dependent.
  • ad hoc to paper The CSL-400h dataset is a valid annotation-free resource and does not introduce biases from controlled recording and generated text.
    This is a new dataset collected by the authors, not released, and is used for the 'extra data' result in Table 3.
invented entities (3)
  • CSL-400h dataset (264,461 videos, 401h)
    purpose: Provides extra annotation-free video-text pairs to scale visual contrastive pretraining and demonstrate data scaling.
    Mentioned with statistics in Table 2 and Supplement B, but no release URL or external validation.
  • CSL-Corpus (737k textual instances)
    purpose: Text corpus for continued pretraining of the LLM in the sign language domain.
    Collected from books, webpages, and GPT-4 generated gloss-text pairs; not released.
  • Word-level visual language token
    purpose: Intermediate representation extracted by local attention transformer, mapped to LLM token space; the paper's claimed innovation.
    Validated only via ablations, not by direct comparison to sign language linguistics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLaVA-SLT: Visual Language Tuning for Sign Language Translation." pith.science (2026). https://pith.science/paper/FQ4WLK7P

@misc{pith2026241216524,
  author       = {Pith},
  title        = {Pith review of: LLaVA-SLT: Visual Language Tuning for Sign Language Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQ4WLK7P}},
  note         = {Machine review of arXiv:2412.16524}
}
read the original abstract

In the realm of Sign Language Translation (SLT), reliance on costly gloss-annotated datasets has posed a significant barrier. Recent advancements in gloss-free SLT methods have shown promise, yet they often largely lag behind gloss-based approaches in terms of translation accuracy. To narrow this performance gap, we introduce LLaVA-SLT, a pioneering Large Multimodal Model (LMM) framework designed to leverage the power of Large Language Models (LLMs) through effectively learned visual language embeddings. Our model is trained through a trilogy. First, we propose linguistic continued pretraining. We scale up the LLM and adapt it to the sign language domain using an extensive corpus dataset, effectively enhancing its textual linguistic knowledge about sign language. Then, we adopt visual contrastive pretraining to align the visual encoder with a large-scale pretrained text encoder. We propose hierarchical visual encoder that learns a robust word-level intermediate representation that is compatible with LLM token embeddings. Finally, we propose visual language tuning. We freeze pretrained models and employ a lightweight trainable MLP connector. It efficiently maps the pretrained visual language embeddings into the LLM token embedding space, enabling downstream SLT task. Our comprehensive experiments demonstrate that LLaVA-SLT outperforms the state-of-the-art methods. By using extra annotation-free data, it even closes to the gloss-based accuracy.

Figures

Figures reproduced from arXiv: 2412.16524 by the authors.

Figure 1
Figure 1. LLaVA-SLT is a highly scalable LMM framework tamed [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Method overview. We train LLaVA-SLT in three stages. First, we collect sign language corpus and scale up the LLMs to enhance the linguistc capabilities in sign language (Sec. 3.1). Subsequently, we employ a large-scale pretrained text encoder to supervise the visual encoder using both inner and outer contrastive losses (Sec. 3.2). Finally, we connect the pretrained visual and language models via a lightweight MLP co… view at source ↗
Figure 3
Figure 3. Wordcloud illustration of the CSL-Corpus dataset. abling learning from both instructional materials and textual representations of sign language itself. The dataset includes a large-scale collection of over 20k textual sign language instances consisting of glosses and the corresponding texts. This data is supplemented with materials gathered from the internet, such as webpage entries and books about sign lan￾guage, … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Hierarchical visual architecture. The visual represen￾tation is explicitly separated into three levels: frame level, word level, and sentence level. the seamless integration of a pretrained visual language model with a textual language model, thereby enhancing the mode…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Attention-Steered Vision-Language Models for Sign Language Translation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    AttnSign adds spatial attention supervision and motion-cadence reinforcement learning to a VLM, improving sign language translation accuracy on How2Sign and OpenASL.

  2. Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Sign Language QA benchmarks are introduced from PHOENIX14T and CSL-Daily via template-generated questions, and a question-conditioned baseline outperforms video-language and cascaded baselines.

  3. EmoSign: A Multimodal Dataset for Understanding Emotions in American Sign Language

    cs.CV 2025-05 conditional novelty 6.0 of 10

    EmoSign is a 200-clip American Sign Language video dataset with native-signer sentiment and emotion labels plus baseline multimodal LLM results showing poor visual-only emotion recognition.

  4. Beyond Words: AuralLLM and SignMST-C for Sign Language Production and Bidirectional Accessibility

    cs.CV 2025-01 reject novelty 6.0 of 10

    Two new Chinese Sign Language datasets and two models are proposed, with a claimed SOTA on PHOENIX2014-T that is unsupported by released artifacts.

  5. A Survey: Learning Embodied Intelligence from Physical Simulators and World Models

    cs.RO 2025-07 conditional novelty 4.0 of 10

    Embodied intelligence learning is reviewed through the complementary lenses of physical simulators and world models, with a proposed IR-L0 to IR-L4 robot capability taxonomy.

Reference graph

Works this paper leans on

123 extracted references · 57 canonical work pages · cited by 5 Pith papers

  1. [1]

    Bsl-1k: Scaling up co- articulated sign language recognition using mouthing cues

    S Albanie, G Varol, L Momeni, et al. Bsl-1k: Scaling up co- articulated sign language recognition using mouthing cues. In Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI, pages 35–53. Springer International Publishing, 2020. 2

    2020

  2. [2]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 7

    arXiv 2023

  3. [3]

    chinese sign language recognition based on surface elec- tromyography and motion information

    Wenyu Li, Zhizeng Luo, Wenguo Li, and Xugang Xi. chinese sign language recognition based on surface elec- tromyography and motion information. PLOS ONE , 18 (12):1–15, 2023. 3

    2023

  4. [4]

    中国手语（上下修订版）.华夏出版社,

    中国聋人协会. 中国手语（上下修订版）.华夏出版社,

  5. [5]

    国家通用手语词典.华夏出版社, 2019

    中国聋人协会. 国家通用手语词典.华夏出版社, 2019. 3

    2019

  6. [6]

    中国手语动词研究

    倪兰. 中国手语动词研究. 2015. 3

    2015

  7. [7]

    中国手语语料库高频词初步分析及标注探讨

    刘学达. 中国手语语料库高频词初步分析及标注探讨. Master’s thesis, 上海外国语大学, 2022. 3

    2022

  8. [8]

    中国手语中的“指点”手势研究

    吕会华. 中国手语中的“指点”手势研究. 绥化学院学报, 37(07):8–13, 2017. 3

    2017

Show all 123 references
  1. [9]

    张吉生. 上海手语音系. 2019. 3

    2019

  2. [10]

    中国手语一般疑问句中疑问手控标记研究

    林皓. 中国手语一般疑问句中疑问手控标记研究. 语言 研究集刊, (01):241–255+379, 2018. 3

    2018

  3. [11]

    潘一. 手语会话. 高等教育出版社, 2005. 3

    2005

  4. [12]

    中国手语音系学中的音节结构研究

    王晓霞. 中国手语音系学中的音节结构研究. Master’s thesis, 西安外国语大学, 2020. 3

    2020

  5. [13]

    手语基础教程.华东师范大学出版社, 2015

    郑璇. 手语基础教程.华东师范大学出版社, 2015. 3

    2015

  6. [14]

    Lora learns less and forgets less

    Dan Biderman, Jacob Portes, Jose Javier Gonzalez Or- tiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Fran- kle, et al. Lora learns less and forgets less. arXiv preprint arXiv:2405.09673, 2024. 4

    2024 arXiv

  7. [15]

    Making themselves heard: Chinese sign language & deaf china online

    Deaflepuff Blog. Making themselves heard: Chinese sign language & deaf china online. https://deaflepuff. tumblr . com / post / 142025989481 / making - themselves- heard- chinese- sign- language,

  8. [16]

    Csl, the revolutionary chinese sign language

    Sign Language Blogs. Csl, the revolutionary chinese sign language. https : / / signlanguage . blog / chinese- sign- language- csl/, 2024. Accessed: November 20, 2024. 3

    2024

  9. [17]

    Neural sign lan- guage translation

    N C Camgoz, S Hadfield, O Koller, et al. Neural sign lan- guage translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7784– 7793, 2018. 1, 2, 4, 6

    2018

  10. [18]

    Sign language transformers: Joint end-to-end sign language recognition and translation

    N C Camgoz, O Koller, S Hadfield, et al. Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 10023–10033, 2020. 1, 2, 6

    2020

  11. [19]

    Idsl: Unleashing dreams and protecting rights of the hearing-impaired

    CGTN. Idsl: Unleashing dreams and protecting rights of the hearing-impaired. https : / / news . cgtn . com/news/2019- 09- 23/International- Day- of - Sign - Languages - Unleashing - dreams - and - protecting - rights - of - the - hearing - impaired- K6HMsDJxfi/index.html , 201...

    2019

  12. [20]

    Idsl: China’s sign language history dates back to tang dynasty period

    CGTN. Idsl: China’s sign language history dates back to tang dynasty period. https : / / news . cgtn . com / news / 2019 - 09 - 23 / IDSL - China - s-sign-language-history-dates-back-to- Tang - Dynasty - period - KamfGp0Q1y / index . html, 2019. Accessed: November 20, 2024. 3

    2019

  13. [21]

    Signvtcl: Multi- modal continuous sign language recognition enhanced by visual-textual contrastive learning

    H Chen, J Wang, Z Guo, et al. Signvtcl: Multi- modal continuous sign language recognition enhanced by visual-textual contrastive learning. arXiv preprint arXiv:2401.11847, 2024. 2

    2024 arXiv

  14. [22]

    Ll3da: Visual interactive instruction tuning for omni-3d understanding reasoning and planning

    Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, and Tao Chen. Ll3da: Visual interactive instruction tuning for omni-3d understanding reasoning and planning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  15. [23]

    A simple multi-modality transfer learning baseline for sign language translation

    Y Chen, F Wei, X Sun, et al. A simple multi-modality transfer learning baseline for sign language translation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 5120–5130, 2022. 1, 2, 6

    2022

  16. [24]

    Two-stream network for sign language recognition and translation

    Y Chen, R Zuo, F Wei, et al. Two-stream network for sign language recognition and translation. Advances in Neural Information Processing Systems , 35:17043–17056, 2022. 1, 2, 6

    2022

  17. [25]

    Factorized learn- ing assisted with large language model for gloss-free sign language translation

    Zhigang Chen, Benjia Zhou, Jun Li, Jun Wan, Zhen Lei, Ning Jiang, Quan Lu, and Guoqing Zhao. Factorized learn- ing assisted with large language model for gloss-free sign language translation. In Proceedings of the 2024 Joint In- ternational Conference on Computational Linguist...

    2024

  18. [26]

    Fully convolu- tional networks for continuous sign language recognition

    K L Cheng, Z Yang, Q Chen, et al. Fully convolu- tional networks for continuous sign language recognition. In Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIV, pages 697–714. Springer International Publishing,

    2020

  19. [27]

    中国手语定中短语的语序

    吕会华, 王红英. 中国手语定中短语的语序. 中国听力语 言康复科学杂志, 15(06):459–463, 2017. 3

    2017

  20. [28]

    论中 国手语的分类词谓语

    姚登峰, 江铭虎,张荣兴, 阿布都克力木·阿布力孜. 论中 国手语的分类词谓语. 中文信息学报, 32(03):1–8, 2018. 3

    2018

  21. [29]

    中国手语基本手势语 义知识组织研究.情报科学, 42(03):118–128, 2024

    张艳琼, 朱兆松,张胜伟, 赵晓驰. 中国手语基本手势语 义知识组织研究.情报科学, 42(03):118–128, 2024. 3

    2024

  22. [30]

    中国手语相同音系参数下的词义拓展和相 关性构建

    文旭,曹阳. 中国手语相同音系参数下的词义拓展和相 关性构建. 西北工业大学学报(社会科学版), (01):77–84,

  23. [31]

    专访上海大学倪 兰教授：语言学与手语识别技术的融合突破，解锁 交 流 障 碍 - gair live

    Tencent Cloud Developer Community. 专访上海大学倪 兰教授：语言学与手语识别技术的融合突破，解锁 交 流 障 碍 - gair live. https://cloud.tencent. com/developer/article/2397557 , 2024. Ac- cessed: November 20, 2024. 3 9

    2024

  24. [32]

    手语（聋人交流方式-百度百 科

    Baidu Baike Contributors. 手语（聋人交流方式-百度百 科. https://baike.baidu.com/item/%E6%89% 8B%E8%AF%AD/67906?fr=ge_ala , 2024. Accessed: November 20, 2024. 3

    2024

  25. [33]

    Is american sign language differ- ent from chinese sign language? https : / / www

    Quora Contributors. Is american sign language differ- ent from chinese sign language? https : / / www . quora . com / Is - American - Sign - Language - different - from - Chinese - Sign - Language ,

  26. [34]

    中国手语 - 维基百科，自由 的百 科全书

    Wikipedia Contributors. 中国手语 - 维基百科，自由 的百 科全书. https://zh.wikipedia.org/zh- hans/%E4%B8%AD%E5%9C%8B%E6%89%8B%E8%AA% 9E, 2024. Accessed: November 20, 2024. 3

    2024

  27. [35]

    Accessed: November 20, 2024. 3

    2024

  28. [36]

    手语 - 维基百科，自由的百科 全书

    Wikipedia Contributors. 手语 - 维基百科，自由的百科 全书. https://zh.wikipedia.org/wiki/%E6% 89%8B%E8%AA%9E , 2024. Accessed: November 20,

    2024

  29. [37]

    Chinese sign language - wikipedia

    Wikipedia Contributors. Chinese sign language - wikipedia. https://en.wikipedia.org/wiki/Chinese_ Sign_Language, 2024. Accessed: November 20, 2024. 3

    2024

  30. [38]

    Asl citizen: A community-sourced dataset for advancing isolated sign lan- guage recognition

    A Desai, L Berger, F Minakov, et al. Asl citizen: A community-sourced dataset for advancing isolated sign lan- guage recognition. Advances in Neural Information Pro- cessing Systems, 36, 2024. 2

    2024

  31. [39]

    A deep neural framework for continuous sign language recognition by iterative training

    R Cui, H Liu, and C Zhang. A deep neural framework for continuous sign language recognition by iterative training. IEEE Transactions on Multimedia, 21(7):1880–1891, 2019. 2

    2019

  32. [40]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    arXiv

  33. [41]

    How2sign: A large- scale multimodal dataset for continuous american sign lan- guage

    A Duarte, S Palaskar, L Ventura, et al. How2sign: A large- scale multimodal dataset for continuous american sign lan- guage. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2735– 2744, 2021. 1, 2

    2021

  34. [42]

    Extensions of the sign language recognition and translation corpus rwth-phoenix- weather

    J Forster, C Schmidt, O Koller, et al. Extensions of the sign language recognition and translation corpus rwth-phoenix- weather. In LREC, pages 1911–1916, 2014. 2, 4

    1911

  35. [43]

    Chatpose: Chatting about 3d human pose

    Y Feng, J Lin, S K Dwivedi, et al. Chatpose: Chatting about 3d human pose. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2093–2103, 2024. 3

    2024

  36. [44]

    A chinese sign language recognition system based on sofm/srn/hmm.Pattern Recog- nition, 37(12):2389–2402, 2004

    W Gao, G Fang, D Zhao, et al. A chinese sign language recognition system based on sofm/srn/hmm.Pattern Recog- nition, 37(12):2389–2402, 2004. 2

    2004

  37. [45]

    中 国 手 语/chinese sign language - non-mandarin chinese - chinese-forums

    Chinese Forums. 中 国 手 语/chinese sign language - non-mandarin chinese - chinese-forums. https : //www.chinese- forums.com/forums/topic/ 50579- %E4%B8%AD%E5%9B%BD%E6%89%8B%E8% AF%ADchinese-sign-language/ , 2024. Accessed: November 20, 2024. 3

    2024

  38. [46]

    Onellm: One framework to align all modalities with language

    Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, and Xi- angyu Yue. Onellm: One framework to align all modalities with language. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 265...

    2024

  39. [47]

    Llms are good sign language translators

    J Gong, L G Foo, Y He, et al. Llms are good sign language translators. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18362– 18372, 2024. 1, 2, 6

    2024

  40. [48]

    Lora: Low- rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low- rank adaptation of large language models. In International Conference on Learning Representations, 202. 4

  41. [49]

    Ma-lmm: Memory-augmented large multimodal model for long-term video understanding

    B He, H Li, Y K Jang, et al. Ma-lmm: Memory-augmented large multimodal model for long-term video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13504–13514, 2024. 3

    2024

  42. [50]

    Self-emphasizing network for continuous sign language recognition

    L Hu, L Gao, Z Liu, et al. Self-emphasizing network for continuous sign language recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 854– 862, 2023. 2

    2023

  43. [51]

    Signbert: Pre-training of hand-model-aware representation for sign language recog- nition

    H Hu, W Zhao, W Zhou, et al. Signbert: Pre-training of hand-model-aware representation for sign language recog- nition. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 11087–11096, 2021. 2

    2021

  44. [52]

    Video-based sign lan- guage recognition without temporal segmentation

    J Huang, W Zhou, Q Zhang, et al. Video-based sign lan- guage recognition without temporal segmentation. In Pro- ceedings of the AAAI Conference on Artificial Intelligence,

  45. [53]

    Continuous sign language recognition with correlation network

    L Hu, L Gao, Z Liu, et al. Continuous sign language recognition with correlation network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2529–2539, 2023a. 2

  46. [54]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186, 2024. 4

    2024 arXiv

  47. [55]

    Audiogpt: Understanding and generating speech, music, sound, and talking head

    R Huang, M Li, D Yang, et al. Audiogpt: Understanding and generating speech, music, sound, and talking head. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 23802–23804, 2024. 3

    2024

  48. [56]

    Llasa: Large multimodal agent for human activity analysis through wearable sensors.arXiv preprint arXiv:2406.14498, 2024

    Sheikh Asif Imran, Mohammad Nur Hossain Khan, Subrata Biswas, and Bashima Islam. Llasa: Large multimodal agent for human activity analysis through wearable sensors.arXiv preprint arXiv:2406.14498, 2024. 3, 4

    2024

  49. [57]

    A dataset for linguistic understanding, visual evaluation, and recogni- tion of sign languages: The k-rsl

    A Imashev, M Mukushev, V Kimmelman, et al. A dataset for linguistic understanding, visual evaluation, and recogni- tion of sign languages: The k-rsl. InProceedings of the 24th Conference on Computational Natural Language Learning, pages 631–640, 2020. 2

    2020

  50. [58]

    Ms-asl: A large-scale dataset and benchmark for understanding american sign language

    H R V Joze and O Koller. Ms-asl: A large-scale dataset and benchmark for understanding american sign language. arXiv preprint arXiv:1812.01053, 2018. 2 10

    2018 arXiv

  51. [59]

    Signclip: Connecting text and sign language by contrastive learning

    Zifan Jiang, Gerard Sant, Amit Moryossef, Mathias M ¨uller, Rico Sennrich, and Sarah Ebling. Signclip: Connecting text and sign language by contrastive learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9171–9193, 2024. 1

    2024

  52. [60]

    Deep sign: Hybrid cnn- hmm for continuous sign language recognition

    O Koller, S Zargaran, H Ney, et al. Deep sign: Hybrid cnn- hmm for continuous sign language recognition. In BMVC, pages 136.1–136.12, 2016. 2

    2016

  53. [61]

    Neural sign language translation based on human keypoint estimation

    S K Ko, C J Kim, H Jung, et al. Neural sign language translation based on human keypoint estimation. Applied Sciences, 9(13):2683, 2019. 2

    2019

  54. [62]

    Transferring cross-domain knowl- edge for video sign language recognition

    D Li, X Yu, C Xu, et al. Transferring cross-domain knowl- edge for video sign language recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6205–6214, 2020. 2

    2020

  55. [63]

    Re-sign: Re-aligned end- to-end sequence modelling with deep recurrent cnn-hmms

    O Koller, S Zargaran, and H Ney. Re-sign: Re-aligned end- to-end sequence modelling with deep recurrent cnn-hmms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4297–4305, 2017a. 2

  56. [64]

    Tspnet: Hierarchical feature learn- ing via temporal semantic pyramid for sign language trans- lation

    D Li, C Xu, X Yu, et al. Tspnet: Hierarchical feature learn- ing via temporal semantic pyramid for sign language trans- lation. In Advances in Neural Information Processing Sys- tems, pages 12034–12045, 2020b. 1, 2, 6

  57. [65]

    Word-level deep sign lan- guage recognition from video: A new large-scale dataset and methods comparison

    D Li, C Rodriguez, X Yu, et al. Word-level deep sign lan- guage recognition from video: A new large-scale dataset and methods comparison. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 1459–1469, 2020a. 2

  58. [66]

    Llava-next-interleave: Tack- ling multi-image, video, and 3d in large multimodal mod- els

    F Li, R Zhang, H Zhang, et al. Llava-next-interleave: Tack- ling multi-image, video, and 3d in large multimodal mod- els. arXiv preprint arXiv:2407.07895, 2024. 3

    2024 arXiv

  59. [68]

    Omg: Towards open-vocabulary motion generation via mixture of controllers

    Han Liang, Jiacheng Bao, Ruichi Zhang, Sihan Ren, Yuecheng Xu, Sibei Yang, Xin Chen, Jingyi Yu, and Lan Xu. Omg: Towards open-vocabulary motion generation via mixture of controllers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4...

    2024

  60. [69]

    Videochat: Chat-centric video understanding

    K C Li, Y He, Y Wang, et al. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355, 2023. 3

    2023 arXiv

  61. [70]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. 6

    2004

  62. [71]

    Video-llava: Learning united visual rep- resentation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual rep- resentation by alignment before projection. arXiv preprint arXiv:2311.10122, 2023. 2, 3, 5

    2023 arXiv

  63. [72]

    Gloss-free end-to-end sign language translation

    Kezhou Lin, Xiaohan Wang, Linchao Zhu, Ke Sun, Bang Zhang, and Yi Yang. Gloss-free end-to-end sign language translation. arXiv preprint arXiv:2305.12876, 2023. 1

    2023 arXiv

  64. [73]

    Chathuman: Language-driven 3d human understanding with retrieval-augmented tool rea- soning

    J Lin, Y Feng, W Liu, et al. Chathuman: Language-driven 3d human understanding with retrieval-augmented tool rea- soning. arXiv preprint arXiv:2405.04533, 2024. 3

    2024 arXiv

  65. [74]

    Visual instruction tuning

    H Liu, C Li, Q Wu, et al. Visual instruction tuning. In Advances in Neural Information Processing Systems, 2024. 2, 3, 5

    2024

  66. [75]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 3, 4, 5

    2024

  67. [76]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 4

    2017 arXiv

  68. [77]

    Multilingual denoising pre-training for neu- ral machine translation

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neu- ral machine translation. Transactions of the Association for Computational Linguistics, 2020. 5, 6

    2020

  69. [78]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    M Maaz, H Rasheed, S Khan, et al. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424, 2023. 3

    2023 arXiv

  70. [79]

    Macaw-llm: Multi-modal language model- ing with image, audio, video, and text integration

    Chenyang Lyu, Minghao Wu, Longyue Wang, Xinting Huang, Bingshuai Liu, Zefeng Du, Shuming Shi, and Zhaopeng Tu. Macaw-llm: Multi-modal language model- ing with image, audio, video, and text integration. arXiv preprint arXiv:2306.09093, 2023. 3

    2023 arXiv

  71. [80]

    手语翻译 - 学习手语 - 手语图 解 - 手语查询

    Sign Language Online. 手语翻译 - 学习手语 - 手语图 解 - 手语查询. https://shouyu.bmcx.com/, 2024. Accessed: November 20, 2024. 3

    2024

  72. [81]

    Visual alignment constraint for continuous sign language recognition

    Y Min, A Hao, X Chai, et al. Visual alignment constraint for continuous sign language recognition. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 11542–11551, 2021. 2

    2021

  73. [82]

    Multimodal sign lan- guage recognition via temporal deformable convolutional sequence learning

    K Papadimitriou and G Potamianos. Multimodal sign lan- guage recognition via temporal deformable convolutional sequence learning. In Interspeech, pages 2752–2756, 2020. 2

    2020

  74. [83]

    Dinov2: Learning robust visual features with- out supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V V o, Marc Szafraniec, Vasil Khalidov, Pierre Fer- nandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El- Nouby, et al. Dinov2: Learning robust visual features with- out supervision. Transactions on Machine Learning Re- s...

    2023

  75. [84]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...

    2021

  76. [85]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311– 318, 2002. 6

    2002

  77. [86]

    The influence of chinese characters on chinese sign language

    Tianyu Ren, Dengfeng Yao, Chaoran Yang, and Xinchen Kang. The influence of chinese characters on chinese sign language. 23(1), 2024. 3

    2024

  78. [87]

    Zero: Memory optimizations toward training trillion parameter models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Confer- ence for High Performance Computing, Networking, Stor- age and Analysis, pages 1–16. IEEE, 2020. 4

    2020

  79. [88]

    Autsl: A large scale multi- modal turkish sign language dataset and baseline methods

    O M Sincan and H Y Keles. Autsl: A large scale multi- modal turkish sign language dataset and baseline methods. IEEE Access, 8:181340–181355, 2020. 2

    2020

  80. [89]

    Connaissez-vous la langue des signes chinoise? un guide complet sur 中 国 手 语

    LTL Language School. Connaissez-vous la langue des signes chinoise? un guide complet sur 中 国 手 语. https : / / ltl - school . fr / langue - des - 11 signes-chinoise/ , 2024. Accessed: November 20,

    2024

  81. [90]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024. 4

    2024

  82. [91]

    Include: A large scale dataset for indian sign language recognition

    A Sridhar, R G Ganesan, P Kumar, et al. Include: A large scale dataset for indian sign language recognition. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1366–1375, 2020. 2

    2020

  83. [92]

    Youtube- asl: A large-scale, open-domain american sign language- english parallel corpus

    Dave Uthus, Garrett Tanzer, and Manfred Georg. Youtube- asl: A large-scale, open-domain american sign language- english parallel corpus. Advances in Neural Information Processing Systems, 36, 2024. 1

    2024

  84. [93]

    Pose-based sign language recognition using gcn and bert

    A Tunga, S V Nuthalapati, and J Wachs. Pose-based sign language recognition using gcn and bert. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 31–40, 2021. 2

    2021

  85. [94]

    Chinese sign language: History, grammar

    Vaia. Chinese sign language: History, grammar. https: / / www . vaia . com / en - us / explanations / chinese / chinese - grammar / chinese - sign - language/, 2024. Accessed: November 20, 2024. 3

    2024

  86. [95]

    Youtube-asl: A large- scale, open-domain american sign language-english paral- lel corpus

    D Uthus, G Tanzer, and M Georg. Youtube-asl: A large- scale, open-domain american sign language-english paral- lel corpus. Advances in Neural Information Processing Sys- tems, 36, 2024. 2

    2024

  87. [96]

    Stochastic transformer networks with linear competing units: Appli- cation to end-to-end sl translation

    A V oskou, K P Panousis, D Kosmopoulos, et al. Stochastic transformer networks with linear competing units: Appli- cation to end-to-end sl translation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11946–11955, 2021. 2

    2021

  88. [97]

    Signum database: Video cor- pus for signer-independent continuous sign language recog- nition

    U von Agris and K F Kraiss. Signum database: Video cor- pus for signer-independent continuous sign language recog- nition. In 4th Workshop on the Representation and Process- ing of Sign Languages: Corpora and Sign Language Tech- nologies, pages 243–246, 2010. 2

    2010

  89. [98]

    What are the types of sign lan- guage? asl, bsl, & more

    wikiHow Contributors. What are the types of sign lan- guage? asl, bsl, & more. https://www.wikihow. com/Types-of-Sign-Language , 2024. Accessed: November 20, 2024. 3

    2024

  90. [99]

    Isolated sign language recognition with grassmann covariance matrices

    H Wang, X Chai, X Hong, et al. Isolated sign language recognition with grassmann covariance matrices. ACM Transactions on Accessible Computing (TACCESS) , 8(4): 1–21, 2016. 2

    2016

  91. [100]

    Sign2gpt: Lever- aging large language models for gloss-free sign language translation

    R Wong, N C Camgoz, and R Bowden. Sign2gpt: Lever- aging large language models for gloss-free sign language translation. arXiv preprint arXiv:2405.04164, 2024. 2, 3, 4

    2024 arXiv

  92. [101]

    Sign2gpt: Leveraging large language models for gloss-free sign language translation

    Ryan Wong, Necati Cihan Camgoz, and Richard Bowden. Sign2gpt: Leveraging large language models for gloss-free sign language translation. In The Twelfth International Conference on Learning Representations. 6, 3

  93. [102]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024. 4

    2024 arXiv

  94. [103]

    What languages are spoken in china? https : / / www

    WorldAtlas. What languages are spoken in china? https : / / www . worldatlas . com / articles / what - languages - are - spoken - in - china . html, 2024. Accessed: November 20, 2024. 3

    2024

  95. [104]

    Cross-modality data augmentation for end-to- end sign language translation

    Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, and Hui Xiong. Cross-modality data augmentation for end-to- end sign language translation. In The 2023 Conference on Empirical Methods in Natural Language Processing. 1

    2023

  96. [105]

    Sign language translation with iterative prototype

    Huijie Yao, Wengang Zhou, Hao Feng, Hezhen Hu, Hao Zhou, and Houqiang Li. Sign language translation with iterative prototype. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 15592– 15601, 2023. 1

    2023

  97. [106]

    Gloss attention for gloss-free sign language translation

    A Yin, T Zhong, L Tang, et al. Gloss attention for gloss-free sign language translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2551–2562, 2023. 2, 6

    2023

  98. [107]

    Improving gloss-free sign language transla- tion by reducing representation density

    Jinhui Ye, Xing Wang, Wenxiang Jiao, Junwei Liang, and Hui Xiong. Improving gloss-free sign language transla- tion by reducing representation density. arXiv preprint arXiv:2405.14312, 2024. 2, 3, 4, 5, 6, 7

    2024 arXiv

  99. [108]

    Sltunet: A simple unified model for sign language translation

    Biao Zhang, Mathias M ¨uller, and Rico Sennrich. Sltunet: A simple unified model for sign language translation. In The Eleventh International Conference on Learning Repre- sentations. 1, 6

  100. [109]

    Better sign language transla- tion with stmc-transformer

    Kayo Yin and Jesse Read. Better sign language transla- tion with stmc-transformer. In Proceedings of the 28th International Conference on Computational Linguistics , pages 5975–5989. International Committee on Computa- tional Linguistics, 2020. 2

    2020

  101. [110]

    Chinese sign language recognition with adaptive hmm

    J Zhang, W Zhou, C Xie, et al. Chinese sign language recognition with adaptive hmm. In 2016 IEEE Interna- tional Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2016. 2

    2016

  102. [111]

    Video-llama: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Sys- tem Demonstrations, pages 543–553, 2023. 2, 3, 4

    2023

  103. [112]

    Svit: Scaling up visual instruction tuning

    Bo Zhao, Boya Wu, Muyang He, and Tiejun Huang. Svit: Scaling up visual instruction tuning. arXiv preprint arXiv:2307.04087, 2023. 4

    2023 arXiv

  104. [113]

    Video instruction tuning with synthetic data

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713, 2024. 2, 3

    2024 arXiv

  105. [114]

    Conditional variational au- toencoder for sign language translation with cross-modal alignment

    R Zhao, L Zhang, B Fu, et al. Conditional variational au- toencoder for sign language translation with cross-modal alignment. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, pages 19643–19651, 2024. 2

    2024

  106. [115]

    Conditional sentence gen- eration and cross-modal reranking for sign language trans- lation

    J Zhao, W Qi, W Zhou, et al. Conditional sentence gen- eration and cross-modal reranking for sign language trans- lation. In IEEE Transactions on Multimedia, pages 2662– 2672, 2021. 2, 6

    2021

  107. [116]

    Gloss-free sign language translation: Improving from visual-language pretraining

    Benjia Zhou, Zhigang Chen, Albert Clap ´es, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. Gloss-free sign language translation: Improving from visual-language pretraining. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2087...

    2023

  108. [117]

    Cvt-slr: Contrastive visual- textual transformation for sign language recognition with variational alignment

    J Zheng, Y Wang, C Tan, et al. Cvt-slr: Contrastive visual- textual transformation for sign language recognition with variational alignment. In Proceedings of the IEEE/CVF 12 Conference on Computer Vision and Pattern Recognition , pages 23141–23150, 2023. 2

    2023

  109. [118]

    Improv- ing sign language translation with monolingual data by sign back-translation

    Hao Zhou, Wengang Zhou, Weizhen Qi, et al. Improv- ing sign language translation with monolingual data by sign back-translation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 1316–1325, 2021. 1, 2, 4, 6

    2021

  110. [119]

    Gloss-free sign language translation: Improving from visual-language pretraining

    B Zhou, Z Chen, A Clap ´es, et al. Gloss-free sign language translation: Improving from visual-language pretraining. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20871–20881, 2023. 2

    2023

  111. [120]

    Spatial-temporal multi- cue network for sign language recognition and translation

    H Zhou, W Zhou, Y Zhou, et al. Spatial-temporal multi- cue network for sign language recognition and translation. IEEE Transactions on Multimedia, 24:768–779, 2021b. 2

  112. [121]

    Spatial-temporal multi-cue network for sign language recognition and translation

    Hao Zhou, Wengang Zhou, Yun Zhou, and Houqiang Li. Spatial-temporal multi-cue network for sign language recognition and translation. IEEE Transactions on Multi- media, 2021. 1

    2021

  113. [122]

    Improving continuous sign lan- guage recognition with consistency constraints and signer removal

    R Zuo and B Mak. Improving continuous sign lan- guage recognition with consistency constraints and signer removal. ACM Transactions on Multimedia Computing, Communications and Applications, 20(6):1–25, 2024. 2

    2024

  114. [123]

    Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models

    D Zhu, J Chen, X Shen, et al. Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models. arXiv preprint arXiv:2304.10592, 2023. 3

    2023 arXiv

  115. [125]

    Natural language-assisted sign language recognition

    R Zuo, F Wei, and B Mak. Natural language-assisted sign language recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14890–14900, 2023. 2 13 LLaV A-SLT: Visual Language Tuning for Sign Language Translation Supplementary Mat...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.