Generative Sign-description Prompts with Multi-positive Contrastive Learning for Sign Language Recognition

Huizhou Chen; Kang Liu; Qiguang Miao; Siyu Liang; Wentian Xin; Xujie Liu; Yunan Li

REVIEW 4 major objections 3 minor 44 references

Generative Sign-description Prompts with Multi-positive Contrastive Learning for Sign Language Recognition

T0 review · 4 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-written sign descriptions push sign recognition to 97.1 percent

desk verdict Solid incremental contribution; the multi-positive contrastive formulation is clean, but the headline fused-number ablation and missing artifacts keep confidence conditional. read the letter →

arxiv 2505.02304 v2 pith:LVQGQKXE submitted 2025-05-05 cs.CL cs.CV

Siyu Liang , Yunan Li , Wentian Xin , Huizhou Chen , Xujie Liu , Kang Liu , Qiguang Miao This is my paper · ORCID

classification cs.CLcs.CV

keywords SignLanguageRecognitionContrastiveLearningGenerativeLargeModelModalityFusionRetrieval-AugmentedGenerationSkeleton-BasedMultipartDescription

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that sign language recognition can be improved by having large language models write multipart descriptions of what each sign looks like, then training a skeleton-based recognizer to match those texts. The proposed method, GSP-MC, anchors the language model's output in expert-validated sign dictionaries through retrieval-augmented generation, producing global, synonym, and part-level descriptions for each sign. A dual-encoder architecture aligns skeleton features with all descriptions of the same sign as multiple positives, using a KL-divergence contrastive loss that generalizes single-positive contrastive learning. On the Chinese SLR-500 dataset the method reaches 97.1% accuracy and on the Turkish AUTSL dataset 97.07%, both state-of-the-art results. If correct, this means generative text descriptions can serve as cheap, scalable auxiliary supervision for sign language recognition without adding inference-time cost.

What carries the argument

The load-bearing mechanism is a multi-positive contrastive loss defined over probabilistic matches. For each skeleton feature $s_i$, the true correspondence distribution is $p_i = I_{\text{match}}(s_i,t_i)/\sum_c I_{\text{match}}(s_i,t_c)$, where $I_{\text{match}}$ is 1 when text and skeleton share a label; the model minimizes a symmetric KL divergence between this distribution and the softmax of cosine similarities in both directions, $s\to t$ and $t\to s$. This generalizes standard single-positive contrastive learning and is applied hierarchically: global skeleton features align with global and synonym texts, while five part-pooled skeleton features align with part-specific texts. The texts themselves come from a generative prompt pipeline that uses retrieval-augmented generation over expert sign dictionaries to suppress hallucination and decompose each sign into part-level descriptions.

What would settle it

Replace the frozen CLIP text encoder with a multilingual text encoder, or translate the Chinese and Turkish descriptions into English before encoding, and rerun the same GSP-MC pipeline on SLR-500 and AUTSL. If accuracy does not drop, the cross-lingual success is not due to the original text encoder's multilingual reliability; if the gains vanish, that assumption is confirmed as load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that a sign's meaning can be transferred into a skeleton-based recognizer through text: the authors generate, for every sign label, a global description, synonym variants, and part-specific descriptions covering left hand, right hand, face, mouth, and body, then train a graph-convolution skeleton encoder so that its global and part-pooled features are pulled toward all text embeddings of the same sign and pushed away from others. The text encoder is frozen, and the contrastive objective treats every description sharing a label as a matched positive via a probabilistic assignment distribution, optimized with a symmetric KL divergence. The paper reports state-of-the-art accuracy of 97.1% on Chinese SLR-500 and 97.07% on Turkish AUTSL, with ablations showing that raw LLM descriptions without grounding hurt accuracy, while retrieval-augmented grounding, optimized prompts, synonym variants, and multipart contrastive learning each add improvement. At inference only the skeleton encoder runs, so the gains come without extra computational overhead.

Load-bearing premise

The method assumes the frozen text encoder produces reliable semantic embeddings for Chinese and Turkish sign descriptions even though it was pretrained mostly on English image-text data; if those embeddings are poor, the contrastive alignment cannot carry the reported cross-lingual gains.

Editorial extensions

If this is right

Because the text encoder and part branches are discarded at inference, the model keeps the speed of a plain skeleton recognizer while gaining text-supervised semantics.
Signs with concrete, decomposable descriptions gain most (the paper reports +17.33% for 'ice cream' and +14.67% for 'experience'), so recognition quality becomes tied to description informativeness.
The same pipeline works for Chinese and Turkish, suggesting that expert-grounded generative descriptions are a language-agnostic supervision signal rather than a per-language annotation effort.
Ablations show that ungrounded LLM output hurts accuracy by 0.28%, and grounding in a knowledge base recovers 1.04% beyond the visual baseline, so the method's success depends on retrieval grounding rather than on the LLM alone.
Multi-positive contrastive learning contributes around 1.36% on AUTSL by capturing inter-sign variation, consistent with the claim that many-to-many text-skeleton alignment beats one-positive alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The frozen text encoder is the untested linchpin: since it was pretrained mostly on English image-text data, a multilingual text encoder or English-translated descriptions could plausibly match or beat the reported gains, a comparison the paper does not run.
The class-wise pattern implies a ceiling set by description quality: signs whose generated texts are vague or wrong lose accuracy, so investing in better prompting or human verification for low-gain classes should transfer directly into accuracy gains.
The same multi-positive, part-aware contrastive formulation could apply to other many-to-many vision-language problems, such as medical images with multiple partial reports or action recognition with multiple valid captions, wherever one visual sample has several legitimate text descriptions.
A natural next test is continuous sign language: if part-level text alignment helps isolated signs, it may also help segment or annotate continuous signing, which the authors list as future work.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is a competent, incremental contribution to skeleton-based sign language recognition. Its new piece is pairing retrieval-augmented generation with expert-verified sign language dictionaries to produce multipart descriptions, then using a multi-positive contrastive loss to align skeleton part features to those descriptions. The multi-positive formulation is a clean generalization of CLIP-style single-positive contrastive learning, and the ablations show each component—RAG, optimized prompts, synonym variants, part-level descriptions—adds a modest but consistent gain. The improvements over prior SOTA are small (about 0.2–0.8 points on SLR-500 and AUTSL), which is typical for this crowded benchmark space.

Where it earns credit: the authors actually tested the failure mode of raw LLM generation, showing it hurts accuracy unless anchored to a knowledge base; that is a useful negative result. The attention visualizations are plausible and align with the described mechanism. The writing is clear and the method is reproducible in principle, though no code or data is released. The citation pattern is unremarkable; the few self-citations are to their own prior encoder work, not a problem.

Soft spots, in rough order of importance. First, the headline 97.1% uses a 4-stream fusion (joint, joint_motion, bone, bone_motion), but every ablation is single-stream joint only. The reader cannot tell how much of the fused gain comes from the contrastive text supervision versus the standard multi-stream ensembling trick. Second, no code, no data release, and no error bars; for gains of a few tenths of a point, that limits confidence in the exact numbers. Third, the cross-lingual assumption is real: a frozen CLIP text encoder pre-trained mostly on English image-text pairs is used for Chinese and Turkish descriptions with no control or comparison to a multilingual encoder. Finally, Table 6's keypoint comparison mixes external methods' configurations with different backbones, so the +1.40% 'multipart' column is not an apples-to-apples comparison.

On the stress-test concern: yes, the positives in Eq. (9) are defined by label alone, so the contrastive loss would still cluster same-class samples even if the text embeddings carried no semantic content. But the Table 4 ablation shows raw LLM output without the knowledge base degrades accuracy, which means text content matters beyond mere label pairing. That does not fully isolate the semantic mechanism—a control with random per-class text would be ideal—but it is enough to keep the central claim plausible.

The paper deserves a serious peer review. It is not groundbreaking, but it is honest, well-scoped engineering that advances a practical subfield. The main requests should be code/data release and an ablation of the 4-stream fusion.

Referee Report

4 major / 3 minor

Summary. The manuscript introduces GSP-MC, a method for isolated sign language recognition from skeleton data. It proposes a retrieval-augmented generation (RAG) pipeline that uses expert-validated sign language dictionaries and an LLM to produce global, synonym, and body-part descriptions for each sign, then embeds these descriptions with a frozen CLIP text encoder. A skeleton encoder (CTR-GCN with 87 HR-Net keypoints) is trained jointly with a cross-entropy classification loss and a multi-positive contrastive loss (Eqs. 9-12) that aligns each skeleton feature with all text descriptions sharing the same sign label. Experiments on SLR-500 and AUTSL report state-of-the-art accuracies of 97.1% and 97.07%, respectively, together with ablations over description source, prompt optimization, multipart contrastive learning, and keypoint selection.

Significance. If the empirical results are reproducible, the paper demonstrates a practical way to inject expert-grounded linguistic descriptions as auxiliary supervision for SLR while keeping the inference-time model as a single skeleton encoder. The work is among the first to use generative LLMs for isolated sign language recognition, and the ablation chain in Tables 4-6 provides useful evidence that the knowledge base and part-level alignment contribute to accuracy. The cross-lingual evaluation on Chinese and Turkish is also a strength. However, the central claim that the gains come from the semantic content of the generated descriptions is not yet fully established, the headline 4-stream fusion result is outside the ablation analysis, and the keypoint-selection comparison is not internally controlled. These issues are addressable and do not by themselves invalidate the approach.

major comments (4)

[Section 3.3.1, Eq. (9)] The positive indicator I_match(s,t_i) is defined solely by whether the skeleton and text share the same sign label, and the text encoder is frozen (Section 3.2.3). As a result, L_con in Eq. (10) is mathematically compatible with a supervised contrastive loss over label-defined positives even if the CLIP text embeddings carry no usable semantic content for Chinese or Turkish descriptions. Table 4 shows that replacing KB-grounded texts with raw LLM output degrades accuracy, which suggests content matters, but this is not a decisive control because the hallucinated texts are still label-paired target vectors. To support the paper's stated contribution, please add a control that removes semantic content while preserving label pairing (e.g., random fixed text embeddings assigned per label, or label-consistent shuffled descriptions), and ideally compare the frozen CLIP text encoder with a multilingual text encoder for Chinese and Turkish. This is the load-bearing evidence for the semantic-supervision claim.
[Section 4.2.1, Table 2 versus Section 4.3.2, Table 5] The headline SLR-500 result of 97.1% is obtained with a 4-stream fusion (joint/joint_motion/bone/bone_motion), but the ablation study in Table 5 is reported only on joint data, and no controlled ablation shows the marginal contribution of GSP-MC at the 4-stream level. The text in Section 4.2.2 attributes the AUTSL improvement of 1.36% to the multi-positive contrastive mechanism, but no ablation table supports that number for the fused configuration. Please report a controlled ablation at the exact configuration used for the headline numbers, including the fusion baseline without GSP-MC.
[Section 4.3.3, Table 6] The keypoint-selection analysis compares the proposed 87-keypoint configuration with results from external methods (MASA, CoSign, etc.) that use different backbones, training protocols, and supervision signals. The 'Parts + Multipart' column therefore does not isolate the effect of the keypoint count. The 'all 133' row (59.22%) is especially difficult to interpret without knowing whether it uses the same encoder, preprocessing, and training procedure. Please provide an internal ablation in which the number of keypoints is varied within the same GSP-MC training pipeline.
[Section 3.3.1, Eqs. (7)-(10)] The mathematical presentation of the contrastive objective is too imprecise for reproduction. In Eq. (7), the subscript a is defined as a = sum_{b=1}^{i-1} m_b + j, but the expression q^{s->t}_{ia} is written with an index j in the numerator; Eq. (8), q^{t->s}(t) = q^{s->t}(t)^T, is not a well-defined operation on a distribution; Eq. (9) mixes a per-sample p_i with a matrix in R^{B x M}; and the claim that the global-description case has M = B despite also using synonym variants T_s is not reconciled with the three-term sum in Eq. (11). Please rewrite this section with consistent indices, dimensions, and normalization.

minor comments (3)

[Abstract and Introduction] The claim of being the 'first work to integrate generative LLMs into SLR tasks' is overstated, since reference [32] (Sign2GPT) already integrates an LLM for gloss-free sign language translation. Please qualify the novelty claim to isolated, skeleton-based SLR or to the specific recognition setting addressed here.
[Section 4.3.1, Table 4] The text says the raw LLM output causes a 0.27% accuracy degradation, while Table 4 shows 93.57% versus 93.85%, which is a 0.28% drop. Please make the numbers consistent.
[Figure 6] Figure 6 contains a long string of '/uni...' tokens that appears to be corrupted or unrendered text. If this is intended to present class names or per-class statistics, it should be replaced with readable labels and a proper caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; GSP-MC is an empirically evaluated supervised training scheme with independent test benchmarks.

full rationale

The paper's pipeline is: sign labels are inputs to LLM prompts (Eqs. 3-6) to generate expert-grounded text descriptions; a frozen CLIP text encoder converts them to features; a contrastive loss (Eqs. 9-10) aligns skeleton features to text features, with positive pairs defined by shared labels; and a cross-entropy loss (Eq. 2) trains the classifier. The test accuracies on SLR-500 and AUTSL are measured on held-out signers and are not used to set hyperparameters (alpha = 0.5 and tau = 0.1 are fixed or validation-based). No prediction in the paper is constructed from the quantity it claims to predict: the descriptions are generated only from training labels, and the label-determined positives in Eq. (9) are a standard supervised-contrastive design choice, not a hidden injection of test labels. The ablations (Tables 4-5) show that removing the knowledge base or prompt optimization changes accuracy, indicating that text content, not merely label pairing, affects results. Self-citations [20] and [22] appear only in related-work context and are not load-bearing. No passage asserts a limitation or omitted proof that would change this assessment. Thus no circular step is exhibited by the paper's own equations or citations, and the score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of pose estimation, the semantic quality of CLIP embeddings for Chinese and Turkish, and the accuracy of LLM-generated descriptions. These are reasonable assumptions but not verified, and the cross-lingual text encoding is not discussed. Hyperparameters alpha, tau, and the keypoint selection are hand-chosen and affect results.

free parameters (3)

alpha = 0.5
Weight of the contrastive loss in Eq. (12), empirically set based on validation performance (Section 3.4).
temperature tau = 0.1
Contrastive temperature in Eq. (7), set to 0.1 for all experiments (Section 4.1.2).
keypoint set (87 keypoints, five-part grouping) = 87 keypoints: 15 body, 21 per hand, 10 mouth, 20 face
Chosen after comparing keypoint counts 27, 49, 76, 87; the paper reports highest accuracy with 87 plus multipart contrastive (Table 6, Section 4.3.3).

assumptions (4)

domain assumption HR-Net keypoint estimation produces accurate joint positions for sign language poses.
All skeleton features are derived from these keypoints; errors in pose estimation would degrade recognition (Section 4.1.2).
domain assumption CLIP text encoder gives semantically meaningful embeddings for Chinese and Turkish sign descriptions.
CLIP is trained predominantly on English; the paper does not validate its cross-lingual quality (Section 3.2.3).
domain assumption LLM-generated descriptions, grounded in official sign dictionaries, are accurate and complete for the signs in both datasets.
The contrastive loss treats these texts as ground truth positives; any hallucinated or missing content would mislead alignment (Section 3.2).
ad hoc to paper The five-part anatomical grouping (face, mouth, left hand, right hand, body) matches the part structure of the generated descriptions.
This decomposition is specific to the method and is not derived from a general principle (Section 3.3.2).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Generative Sign-description Prompts with Multi-positive Contrastive Learning for Sign Language Recognition." pith.science (2026). https://pith.science/paper/LVQGQKXE

@misc{pith2026250502304,
  author       = {Pith},
  title        = {Pith review of: Generative Sign-description Prompts with Multi-positive Contrastive Learning for Sign Language Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LVQGQKXE}},
  note         = {Machine review of arXiv:2505.02304}
}

read the original abstract

Sign language recognition (SLR) faces fundamental challenges in creating accurate annotations due to the inherent complexity of simultaneous manual and non-manual signals. To the best of our knowledge, this is the first work to integrate generative large language models (LLMs) into SLR tasks. We propose a novel Generative Sign-description Prompts Multi-positive Contrastive learning (GSP-MC) method that leverages retrieval-augmented generation (RAG) with domain-specific LLMs, incorporating multi-step prompt engineering and expert-validated sign language corpora to produce precise multipart descriptions. The GSP-MC method also employs a dual-encoder architecture to bidirectionally align hierarchical skeleton features with multiple text descriptions (global, synonym, and part level) through probabilistic matching. Our approach combines global and part-level losses, optimizing KL divergence to ensure robust alignment across all relevant text-skeleton pairs while capturing both sign-level semantics and detailed part dynamics. Experiments demonstrate state-of-the-art performance against existing methods on the Chinese SLR500 (reaching 97.1%) and Turkish AUTSL datasets (97.07% accuracy). The method's cross-lingual effectiveness highlight its potential for developing inclusive communication technologies.

Figures

Figures reproduced from arXiv: 2505.02304 by the authors.

**Figure 2.** The overall architecture of GSP-MC method. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Architecture of the customized LLM for sign de [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Generation of multipart texts. 3.2.3 Text Encoder. The text encoding component employs the CLIP text encoder as its backbone 𝐸𝑡 , processing three distinct text modalities: global descriptions 𝑇𝑔, semantically-rich synonym variants 𝑇𝑠 , and fine-grained part-specific t…

**Figure 5.** Figure 5: Attention Visualization Comparison: Skeleton [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Class-wise Performance Gains method versus the baseline CTR-GCN model on SLR-500. The visual-text contrastive learning method produces markedly different impacts across sign categories, revealing important patterns about its operational mechanisms. The substantial 17.…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 21 canonical work pages

[1]

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Jianfeng Gao, Songhao Piao, Ming Zhou, and Hsiao-Wuen Hon. 2020. UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre- Training. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daum...

work page 2020
[2]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Pro- ceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119) . PmLR, Online, 1597–1607. http://proceedin...

work page 2020
[3]

Yuxin Chen, Ziqi Zhang, Chunfeng Yuan, Bing Li, Ying Deng, and Weiming Hu
[4]

Zhiwen Deng, Yuquan Leng, Junkang Chen, Xiang Yu, Yang Zhang, and Qing Gao

work page
[5]

Zhiwen Deng, Yuquan Leng, Jing Hu, Zengrong Lin, Xuerui Li, and Qing Gao

work page
[6]

Gu Dingqian, Wei Dan, Wang Chenhua, Gao Hui, Yu Yuanyuan, Heng Miao, Qiu Bing, and Wu Yongsheng. 2019. Chinese manual alphabet. Technical Report GF 0021–2019. Ministry of Education of the People’s Republic of China, State Lan- guage Commission, China Disabled Persons’ Federation, Beijing, China. National Standard

work page 2019
[7]

Gu Dingqian, Wei Dan, Yang Yang, Wang Chenhua, Yu Yuanyuan, Gao Hui, Wu Yongsheng, Heng Miao, Qiu Bing, Liu Wa, Xu Cong, Wang Yibo, Sun Lianqun, Sun Wanli, and Wang Jian. 2018. Lexicon of Common Expressions in Chinese National Sign Language. Technical Report GF 0020–2018. Ministry of Education of the People’s Republic of China, State Language Commission, ...

work page 2018
[8]

Knowledge-Based Systems 301 (2024), 112288

SML: A Skeleton-based multi-feature learning method for sign language recognition. Knowledge-Based Systems 301 (2024), 112288. doi:10.1016/J.KNOSYS. 2024.112288

arXiv 2024

Show all 44 references

[9]

Girshick

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. 2020. Mo- mentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation / IEEE, 9729–9738. do...

2020
[10]

Hezhen Hu, Weichao Zhao, Wengang Zhou, Yuechen Wang, and Houqiang Li
[11]

Hao-Shu Fang, Jiefeng Li, Hongyang Tang, Chao Xu, Haoyi Zhu, Yuliang Xiu, Yong-Lu Li, and Cewu Lu. 2022. Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time. IEEE transactions on pattern analysis and machine intelligence 45, 6 (2022), 7157–717...

2022
[12]

Eui Jun Hwang, Sukmin Cho, Junmyeong Lee, and Jong C Park. 2024. An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMs. doi:10.48550/ARXIV.2408.10593

2024 doi arXiv
[13]

Intel. 2019. Intel® RealSense™ Technology. https://www.intel.com/content/ www/us/en/architecture-and-technology/realsense-overview.html Accessed on 25/02/2025

2019
[14]

In Proceedings of the IEEE/CVF international conference on computer vision

c. In Proceedings of the IEEE/CVF international conference on computer vision . ICCV, Virtual, 11087–11096. doi:10.1109/ICCV48922.2021.01090

2021
[15]

Jie Huang, Wengang Zhou, Houqiang Li, and Weiping Li. 2018. Attention-based 3D-CNNs for large-vocabulary sign language recognition. IEEE Transactions on Circuits and Systems for Video Technology 29, 9 (2018), 2822–2832. doi:10.1109/ TCSVT.2018.2870740

2018
[17]

Sai Shashank Kalakonda, Shubh Maheshwari, and Ravi Kiran Sarvadevabhatla
[18]

Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, and Yun Fu. 2021. Sign language recognition via skeleton-aware multi-model ensemble. CoRR abs/2110.06161 (2021). arXiv:2110.06161 https://arxiv.org/abs/2110.06161

2021 arXiv
[19]

Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, and Yun Fu. 2021. Skeleton aware multi-modal sign language recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . Computer Vision Foundation / IEEE, 3413–3423. doi:10.1109/CV...

2021
[20]

Siyu Liang, Yunan Li, Yuanyuan Shi, Huizhou Chen, and Qiguang Miao. 2025. Integrated multi-local and global dynamic perception structure for sign language recognition. Pattern Analysis and Applications 28, 2 (2025), 1–14. doi:10.1007/ S10044-024-01403-8

2025
[21]

Kezhou Lin, Xiaohan Wang, Linchao Zhu, Bang Zhang, and Yi Yang. 2024. SKIM: Skeleton-based isolated sign language recognition with part mixing. IEEE Trans- actions on Multimedia 26 (2024), 4271–4280. doi:10.1109/TMM.2023.3321502

2024
[22]

Kang Liu, Zhuoqi Ma, Xiaolu Kang, Yunan Li, Kun Xie, Zhicheng Jiao, and Qiguang Miao. 2025. Enhanced Contrastive Learning with Multi-view Longi- tudinal Data for Chest X-ray Report Generation. CoRR abs/2502.20056 (2025). doi:10.48550/ARXIV.2502.20056 arXiv:2502.20056

2025 doi
[23]

Semih Kavak. 2015. Turkish Sign Language Dictionary . National Education Ministry

2015
[24]

David Laines, Miguel Gonzalez-Mendoza, Gilberto Ochoa-Ruiz, and Gissella Be- jarano. 2023. Isolated sign language recognition based on tree structure skeleton images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 276–284. doi:10.1109/CVP...

2023
[25]

Microsoft. 2017. Developing with Kinect . https://developer.microsoft.com/en- us/windows/kinect/develop Accessed on 25/02/2025

2017
[26]

China Association of the Deaf and Hard of Hearing (Eds.). 2018. Chinese Sign Language (Revised Edition) (Parts 1 & 2) . Huaxia Publishing House

2018
[27]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. CoRR abs/1807.03748 (2018). arXiv:1807.03748 http://arxiv.org/abs/1807.03748

2018 arXiv
[28]

Ruyi Liu, Yi Liu, Mengyao Wu, Wentian Xin, Qiguang Miao, Xiangzeng Liu, and Long Li. 2025. SG-CLR: Semantic representation-guided contrastive learning for self-supervised skeleton-based action recognition. Pattern Recognition 162 (2025), 111377. doi:10.1016/J.PATCOG.2025.111377

2025
[29]

Yuhong Liu, Fei Lu, Xianpeng Cheng, and Ying Yuan. 2024. Asymmetric multi- branch GCN for skeleton-based sign language recognition. Multimedia Tools and Applications 83, 30 (2024), 75293–75319. doi:10.1007/S11042-024-18443-1

2024 doi
[30]

Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. 2019. Deep high-resolution representation learning for human pose estimation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5693–5703. doi:10.1109/ CVPR.2019.00584 MM ’25, October 27–30, 202...

2019
[31]

Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan
[32]

Ryan Wong, Necati Cihan Camgöz, and Richard Bowden. 2024. Sign2GPT: Lever- aging Large Language Models for Gloss-Free Sign Language Translation. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net. ht...

2024
[33]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al . 2021. Learning transferable visual models from natural language su- pervision. In International conference on machine lea...

2021
[34]

Ozge Mercanoglu Sincan and Hacer Yalim Keles. 2020. Autsl: A large scale multi- modal turkish sign language dataset and baseline methods. IEEE Access 8 (2020), 181340–181355. doi:10.1109/ACCESS.2020.3028072

2020
[35]

Tingbing Yan, Wenzheng Zeng, Yang Xiao, Xingyu Tong, Bo Tan, Zhiwen Fang, Zhiguo Cao, and Joey Tianyi Zhou. 2024. Crossglg: Llm guides one-shot skeleton- based 3d action recognition in a cross-level manner. In European Conference on Computer Vision, Vol. 15078. Springer, 113–1...

2024 doi
[36]

Lin Yuan, Zhen He, Qiang Wang, Leiyang Xu, and Xiang Ma. 2022. Skeleton- clip: Recognizing skeleton-based human actions with text prompts. In 2022 8th International Conference on Systems and Informatics (ICSAI) . IEEE, 1–6. doi:10.1109/ICSAI57119.2022.10005459

2022
[37]

Advances in Neural Information Processing Sys- tems 36 (2023), 48382–48402

Stablerep: Synthetic images from text-to-image models make strong vi- sual representation learners. Advances in Neural Information Processing Sys- tems 36 (2023), 48382–48402. http://papers.nips.cc/paper_files/paper/2023/hash/ 971f1e59cd956cc094da4e2f78c6ea7c-Abstract-Conference.html

2023
[38]

Weichao Zhao, Hezhen Hu, Wengang Zhou, Yunyao Mao, Min Wang, and Houqiang Li. 2024. MASA: Motion-aware Masked Autoencoder with Semantic Alignment for Sign Language Recognition. IEEE Transactions on Circuits and Systems for Video Technology 34, 11 (2024), 10793–10804. doi:10.11...

2024
[40]

Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial temporal graph convolu- tional networks for skeleton-based action recognition. In Proceedings of the AAAI conference on artificial intelligence , Vol. 32. doi:10.1609/aaai.v32i1.12328

2018 doi
[41]

Ronglai Zuo, Fangyun Wei, and Brian Mak. 2023. Natural Language-Assisted Sign Language Recognition. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 14890–14900. doi:10.1109/CVPR52729.2023.01430 bv c © 2024 ACM. Licensed under CC-BY. ACM-XXXX-YYYY-ZZZZ

2023
[43]

Huaiwen Zhang, Zihang Guo, Yang Yang, Xin Liu, and De Hu. 2023. C2st: Cross-modal contextualized sequence transduction for continuous sign language recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 21053–21062. doi:10.1109/ICCV51070...

2023
[45]

Weichao Zhao, Hezhen Hu, Wengang Zhou, Jiaxin Shi, and Houqiang Li. 2023. BEST: BERT pre-training for sign language recognition with coupling tokeniza- tion. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 37. 3597–3605. doi:10.1609/AAAI.V37I3.25470

2023 doi
[46]

Weichao Zhao, Wengang Zhou, Hezhen Hu, Min Wang, and Houqiang Li. 2024. Self-supervised representation learning with spatial-temporal consistency for sign language recognition. IEEE Transactions on Image Processing 33 (2024), 4188–4201. doi:10.1109/TIP.2024.3416881

2024
[2023]

In 2023 IEEE International Conference on Multimedia and Expo (ICME)

Action-gpt: Leveraging large-scale language models for improved and gen- eralized action generation. In 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 31–36. doi:10.1109/ICME55011.2023.00014

2023
[2024]

Neurocomputing 572 (2024), 127194

TMS-Net: A multi-feature multi-stream multi-level information sharing network for skeleton-based sign language recognition. Neurocomputing 572 (2024), 127194. doi:10.1016/J.NEUCOM.2023.127194

2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Jianfeng Gao, Songhao Piao, Ming Zhou, and Hsiao-Wuen Hon. 2020. UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre- Training. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daum...

work page 2020

[2] [2]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Pro- ceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119) . PmLR, Online, 1597–1607. http://proceedin...

work page 2020

[3] [3]

Yuxin Chen, Ziqi Zhang, Chunfeng Yuan, Bing Li, Ying Deng, and Weiming Hu

[4] [4]

Zhiwen Deng, Yuquan Leng, Junkang Chen, Xiang Yu, Yang Zhang, and Qing Gao

work page

[5] [5]

Zhiwen Deng, Yuquan Leng, Jing Hu, Zengrong Lin, Xuerui Li, and Qing Gao

work page

[6] [6]

Gu Dingqian, Wei Dan, Wang Chenhua, Gao Hui, Yu Yuanyuan, Heng Miao, Qiu Bing, and Wu Yongsheng. 2019. Chinese manual alphabet. Technical Report GF 0021–2019. Ministry of Education of the People’s Republic of China, State Lan- guage Commission, China Disabled Persons’ Federation, Beijing, China. National Standard

work page 2019

[7] [7]

Gu Dingqian, Wei Dan, Yang Yang, Wang Chenhua, Yu Yuanyuan, Gao Hui, Wu Yongsheng, Heng Miao, Qiu Bing, Liu Wa, Xu Cong, Wang Yibo, Sun Lianqun, Sun Wanli, and Wang Jian. 2018. Lexicon of Common Expressions in Chinese National Sign Language. Technical Report GF 0020–2018. Ministry of Education of the People’s Republic of China, State Language Commission, ...

work page 2018

[8] [8]

Knowledge-Based Systems 301 (2024), 112288

SML: A Skeleton-based multi-feature learning method for sign language recognition. Knowledge-Based Systems 301 (2024), 112288. doi:10.1016/J.KNOSYS. 2024.112288

arXiv 2024