Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Speak2Sign3D: A Multi-modal Pipeline for English Speech to American Sign Language Animation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims a complete pipeline from spoken English to 3D American Sign Language animation, with MarianMT reaching BLEU 0.8923 on an automatically built ASL gloss corpus and cubic spline interpolation achieving MPJPE 0.0429.

desk verdict A competent engineering integration, but the evaluation is circular and the central claim of accurate ASL animation is unsupported. read the letter →

arxiv 2507.06530 v1 pith:IK7FYMN7 submitted 2025-07-09 cs.CV

Kazi Mahathir Rahman , Naveed Imtiaz Nafis , Md. Farhan Sadik , Mohammad Al Rafi , Mehedi Hasan Shahed This is my paper
classification cs.CV
keywords speech-to-signtranslationAmericanSignLanguageglossMarianMT3DanimationkeypointposeestimationcubicsplineinterpolationWLASLBLEUscore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a fully automatic pipeline can take spoken English, transcribe it with Whisper, translate it into American Sign Language gloss with a fine-tuned MarianMT model, and render the gloss as a smooth 3D animation using keypoints extracted from WLASL videos and cubic spline interpolation. The authors report MarianMT achieving BLEU 0.7714 and 0.8923 on English-to-gloss translation, beating T5 and BART, and cubic interpolation achieving MPJPE 0.0429, slightly better than an LSTM baseline. They also construct two datasets: BookGlossCorpus-CG, about 1.3 million English sentences converted to ASL gloss by hand-written grammar rules, and Sign3D-WLASL, keypoint-extracted 3D skeletal motion from real ASL videos. If correct, this matters because it demonstrates a complete speech-to-sign path without requiring paired sign-language video generation.

What carries the argument

The load-bearing mechanism is the pairing of a rule-based ASL gloss grammar with a trainable translator. Algorithm 1 reorders each English clause into the gloss order [time, topic, verb, feelings, wh] and inserts ASL-specific tokens such as # for lexicalized fingerspelling and fs- for fingerspelled proper nouns; BookGlossCorpus-CG is generated by applying this grammar to about 1.3 million BookCorpus sentences. MarianMT is then fine-tuned on the resulting English-gloss pairs, so the grammar defines both the training target and the evaluation reference. The animation side rests on keypoints: 133 body, hand, and face keypoints extracted from WLASL videos by RTMPose3D, ResNet50-H3WB, OpenPifPaf with JointFormer, and MediaPipe, temporally downsampled and then re-expanded with cubic spline interpolation.

What would settle it

Have several fluent ASL signers rate a random sample of BookGlossCorpus-CG glosses and the pipeline's animations for grammaticality and naturalness; if signers reject the gloss order or cannot understand the animations, the central claim fails even if BLEU and MPJPE values look good. Recomputing BLEU against a reference set produced by human ASL translators would also settle whether the reported scores reflect real translation quality.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes a working pipeline in which each stage feeds the next: Whisper converts audio to text, MarianMT translates English text to ASL gloss, a word-mapping step normalizes gloss tokens, and a keypoint animation module maps glosses to 3D sign motion with cubic spline interpolation. Its central quantitative claims are that MarianMT outperforms T5 and BART on English-to-gloss translation (BLEU 0.7714 and 0.8923 versus 0.0184 and 0.0634 for T5, and 0.5577 and 0.7067 for BART), and that cubic spline interpolation yields lower MPJPE (0.0429) than an LSTM-based motion model (0.044). The paper interprets these numbers as evidence that transformer-based translation plus interpolation can generate accurate and temporally smooth sign language animations.

Load-bearing premise

Every translation score rests on the assumption that the hand-written grammar in Algorithm 1 produces valid ASL gloss, because that grammar generates both the training targets and the references used to compute BLEU, and this assumption is never checked against fluent signers or an external ASL standard.

Editorial extensions

If this is right

  • The pipeline offers a viable architecture for speech-to-sign translation that does not require paired sign-language video for training; keypoint extraction plus interpolation is enough to produce animation.
  • MarianMT fine-tuned on a rule-generated gloss corpus becomes the strongest of the three tested translation models, so future English-to-ASL gloss work can use it as a baseline.
  • Cubic spline interpolation can replace a learned motion model when the target motions are relatively consistent, matching or slightly beating an LSTM in joint-position error.
  • The two released datasets, BookGlossCorpus-CG and Sign3D-WLASL, provide resources for training and evaluating gloss translation and 3D sign animation separately.
  • The full chain from audio to animation suggests a deployable accessibility tool that starts from ordinary spoken English input.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported BLEU scores are computed against references produced by the same rule-based grammar that generated the training targets, so they likely measure how consistently the model reproduces Algorithm 1 rather than whether the gloss is understandable to ASL signers.
  • Because no fluent signer evaluation or comparison with human-produced ASL gloss appears in the paper, the claim that the animations are 'accurate' and 'natural' remains an open empirical question.
  • The keypoint animation stage stitches together signs that come from isolated WLASL entries; full-sentence signing also requires coarticulation and grammatical facial expressions, so a sentence-level animation evaluation is a natural next test.
  • If the rule-based gloss order is independently validated by signers, the same corpus-generation approach could be extended to other sign languages by swapping the grammar rules.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Speak2Sign3D, a multimodal pipeline that converts English speech into 3D American Sign Language animations. The pipeline consists of Whisper for speech recognition, MarianMT (compared with T5 and BART) for English-to-ASL gloss translation on a new rule-generated corpus BookGlossCorpus-CG, a cosine-similarity word-mapping step, keypoint extraction from WLASL videos to create a Sign3D-WLASL dataset, and cubic spline interpolation to animate the keypoints. The authors report BLEU scores of 0.7714/0.8923 for MarianMT and MPJPE of 0.0429 for cubic interpolation, and conclude that the system demonstrates accurate and temporally smooth sign language animation.

Significance. If substantiated, an end-to-end speech-to-3D-ASL pipeline would be a valuable accessibility tool, and the paper addresses an underexplored direction relative to sign language recognition. The manuscript also contributes two datasets (BookGlossCorpus-CG and Sign3D-WLASL) and a concrete system architecture. However, the reported evaluation is not sufficient to support the central claim. The BLEU metric is evaluated against references produced by the same hand-written grammar that generated the training targets, making the scores a measure of rule reproduction rather than ASL quality; the MPJPE numbers are presented without units or uncertainty; and no human or external validation is performed. The paper therefore reads as a pipeline proposal with an internal consistency check rather than an evidence-based demonstration of accurate ASL translation.

major comments (4)
  1. [§3.2, §4.1, Table 1] The BLEU evaluation is circular. BookGlossCorpus-CG is created by Algorithm 1, which specifies a fixed ordering (time, topic, verb, feelings, wh) and special tokens (fs-, #). Both the training targets and the test references come from this same rule set, so a model that reproduces Algorithm 1 will trivially achieve high BLEU. The paper does not validate the gloss grammar against fluent ASL signers or an external ASL standard; indeed, §5 lists 'incorporating linguistic rules and syntactic structures to more accurately represent ASL grammar' as future work, which concedes the current grammar is incomplete. Therefore the BLEU scores cannot support the conclusion that the translations are accurate ASL.
  2. [Table 1] The reported BLEU values are internally inconsistent: T5 achieves BLEU-1 = 0.0184 and BLEU-2 = 0.0634. Under standard BLEU computation, BLEU-2 is BP * sqrt(p1 * p2) with p2 ≤ p1 and BP ≤ 1, so BLEU-2 cannot exceed BLEU-1. This inversion suggests a miscalculation or a non-standard metric and calls into question the other BLEU numbers, including the headline MarianMT scores.
  3. [§4.2, Table 2] The MPJPE comparison is under-specified. No units, standard deviations, or number of test samples are given; the difference between 0.044 and 0.0429 is 0.0011 and is likely within noise. Moreover, MPJPE on keypoint regression measures pose estimation error, not whether the resulting motion is grammatical or intelligible ASL. The claim that cubic interpolation provides 'precise motion smoothing' in sign language generation is therefore not established by the reported numbers.
  4. [§3.5, §3.6, §6] The word-mapping stage selects the top-1 cosine-similarity gloss for each token with no validation of the semantic mapping, and the animation stage simply interpolates keypoints from isolated WLASL signs. There is no evidence that the selected gloss sequence forms a grammatical or intelligible ASL utterance, and no evaluation by DHH users or sign-language experts is reported. The conclusion that the pipeline generates 'accurate' animations is thus unsupported by the experiments.
minor comments (5)
  1. [General] The manuscript does not include a reproducibility statement or links to the datasets/code, despite introducing BookGlossCorpus-CG and Sign3D-WLASL.
  2. [§2] There are citation inconsistencies: 'Liu et al. [6]' refers to reference [6], which is Costa et al. on sign language translation, and references [38] and [39] are duplicates of the same paper.
  3. [§4, Figure 5] Figure 1 (model architecture) is never described in the text, and the learning curves in Figure 5(a) are not analyzed.
  4. [Abstract, §3.2] The abstract describes gloss as a 'simplified version of sign language that captures meaning without grammar,' which conflicts with §3.2's claim that the dataset follows ASL grammar rules; this should be clarified.
  5. [General] There are typos and formatting issues: 'M ULTI-MODAL' in the title, 'Sample of RMTPose3D' in §7, and inconsistent naming between 'Sign3D-WLASL' and 'WLASL to Keypoint Dataset'.

Circularity Check

1 steps flagged · score 8.0 of 10

BLEU scores in Table 1 are circular: BookGlossCorpus-CG references are generated by the same Algorithm 1 rules the model is trained to imitate, so high BLEU does not establish ASL accuracy.

  1. fitted input called prediction [Section 3.2 (Algorithm 1), Section 3.4.3, Section 4.1 Table 1]
    "It contains approximately 1.3 million English sentences that have been transformed into ASL gloss using a set of linguistically informed grammar rules. ... This gloss-converted dataset, BookGlossCorpus-CG, is later used to train our MarianMT translation model. ... MarianMT significantly outperforms the others, achieving scores of 0.7714 and 0.8923, indicating better translation accuracy for English to ASL gloss."

    The training targets and the BLEU references are both produced by the same hand-written rule set (Algorithm 1: time, topic, verb, feelings, wh order, fs-/ # tokens). MarianMT is trained with teacher forcing on those outputs, so its 'predictions' are optimized to reproduce Algorithm 1. Computing BLEU against the same rule-generated references therefore measures only how faithfully the model mimics that grammar, not whether the glosses are valid ASL. The paper reports no evaluation by fluent signers or comparison to an independent ASL gloss corpus, and Section 5 lists 'incorporating linguistic rules and syntactic structures to more accurately represent ASL grammar' as future work, conceding the rule set itself is not established ASL.

full rationale

The central derivation chain is: BookCorpus is converted to ASL gloss by Algorithm 1; this synthetic corpus trains MarianMT; BLEU is computed against the same synthetic corpus. By the paper's own text, the evaluation and training references have the identical origin, so a high BLEU score is forced once the model learns the rule-based mapping. No independent human reference or external ASL standard is used, and the future-work section admits that current ASL grammar representation is incomplete. Consequently the claim that the pipeline generates 'accurate' sign language cannot be supported by the reported numbers; only the claim that the model reproduces Algorithm 1 is supported. The animation stage has similarly no external evaluation of sign correctness: MPJPE compares interpolation and LSTM outputs against keypoints extracted by the authors' own pipeline. These are synthetic-reference benchmarks, not validations against real signer behavior. The overall circularity score is high because the paper's headline accuracy result reduces by construction to its own synthetic reference set.

Assumptions & free parameters 3 free parameters · 4 assumptions · 2 invented entities

The central claim rests almost entirely on hand-specified ASL grammar rules and internally extracted keypoints. The free parameters are choices made by the authors, and the two new datasets are the only introduced artifacts; neither has independent evidence because neither is released or externally evaluated.

free parameters (3)
  • ASL gloss grammar rule order = [time, topic, verb, feelings, wh]
    Hand-specified in Algorithm 1; determines every training target and BLEU reference in BookGlossCorpus-CG, so it fully controls the gloss translation score.
  • Frame downsampling interval = every 4th or 8th frame
    Chosen in Section 3.3 to reduce data volume; affects keypoint density and smoothness of Sign3D-WLASL and all animation outputs, with no ablation.
  • Cosine similarity top-1 selection = top-1 gloss
    Word mapping in Section 3.5 selects the highest-similarity ASL gloss; thresholds or fallback behavior are not specified.
assumptions (4)
  • standard math Cubic spline interpolation preserves continuity of position and first/second derivatives
    Invoked in Section 3.6 to justify smooth motion; standard mathematical property.
  • ad hoc to paper ASL can be represented by the gloss ordering [time, topic, verb, feelings, wh] with fs- and # markers
    Algorithm 1 and Section 3.2 assume ASL grammar reduces to this ordering; never validated against fluent signers.
  • domain assumption Off-the-shelf pose estimators on WLASL video yield keypoints sufficient to represent ASL signs
    Section 3.3 assumes 133 keypoints from RTMPose3D, ResNet50-H3WB, OpenPifPaf/JointFormer, and MediaPipe capture the phonology of ASL, including handshape and facial grammar.
  • domain assumption BLEU on gloss sequences measures ASL translation quality
    BLEU counts n-gram overlap with the rule-generated references, which is a proxy only for consistency with the rule system, not for linguistic correctness.
invented entities (2)
  • BookGlossCorpus-CG
    purpose: Training and evaluation corpus for English-to-ASL gloss translation
    Described in Section 3.2; no release URL, so no external handle to verify or reuse the corpus.
  • Sign3D-WLASL keypoint dataset
    purpose: Motion source for 3D ASL animation
    Created in Section 3.3 from WLASL video by pose estimation; not released and no benchmarks, so the keypoint quality cannot be independently assessed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Speak2Sign3D: A Multi-modal Pipeline for English Speech to American Sign Language Animation." pith.science (2026). https://pith.science/paper/IK7FYMN7

@misc{pith2026250706530,
  author       = {Pith},
  title        = {Pith review of: Speak2Sign3D: A Multi-modal Pipeline for English Speech to American Sign Language Animation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IK7FYMN7}},
  note         = {Machine review of arXiv:2507.06530}
}
read the original abstract

Helping deaf and hard-of-hearing people communicate more easily is the main goal of Automatic Sign Language Translation. Although most past research has focused on turning sign language into text, doing the reverse, turning spoken English into sign language animations, has been largely overlooked. That's because it involves multiple steps, such as understanding speech, translating it into sign-friendly grammar, and generating natural human motion. In this work, we introduce a complete pipeline that converts English speech into smooth, realistic 3D sign language animations. Our system starts with Whisper to translate spoken English into text. Then, we use a MarianMT machine translation model to translate that text into American Sign Language (ASL) gloss, a simplified version of sign language that captures meaning without grammar. This model performs well, reaching BLEU scores of 0.7714 and 0.8923. To make the gloss translation more accurate, we also use word embeddings such as Word2Vec and FastText to understand word meanings. Finally, we animate the translated gloss using a 3D keypoint-based motion system trained on Sign3D-WLASL, a dataset we created by extracting body, hand, and face key points from real ASL videos in the WLASL dataset. To support the gloss translation stage, we also built a new dataset called BookGlossCorpus-CG, which turns everyday English sentences from the BookCorpus dataset into ASL gloss using grammar rules. Our system stitches everything together by smoothly interpolating between signs to create natural, continuous animations. Unlike previous works like How2Sign and Phoenix-2014T that focus on recognition or use only one type of data, our pipeline brings together audio, text, and motion in a single framework that goes all the way from spoken English to lifelike 3D sign language animation.

Figures

Figures reproduced from arXiv: 2507.06530 by the authors.

Figure 1
Figure 1. Model Architecture This research aims to develop a comprehensive speech-to-sign language translation system that combines state-of-the-art speech recognition, machine translation, and 3D animation techniques to improve accessibility for the Deaf and Hard of Hearing (DHH) community. The core goal is to evaluate how effectively spoken English can be converted into natural and continuous 3D American Sign Language (ASL)… view at source ↗
Figure 2
Figure 2. Workflow step by step Algorithm 1 ASL Gloss Generation for BookGlossCorpus-CG 1: for sentence in BookCorpus do 2: sentence ← clean_text(sentence) 3: clauses ← split_into_clauses(sentence) 4: for clause in clauses do 5: time ← extract_temporal_words(clause) 6: topic ← extract_topic_nouns(clause) 7: verb ← extract_verbs(clause) 8: feelings ← extract_adjectives_and_adverbs(clause) 9: wh ← extract_wh_questions(clause) 1… view at source ↗
Figure 3
Figure 3. Gloss demo 3.3 Sign3D-WLASL Dataset To support realistic 3D gesture generation, we constructed the WLASL [24] to Keypoint Dataset, which enriches the original WLASL video dataset with detailed 3D skeletal motion data. The WLASL dataset includes 1,983 carefully curated videos that capture isolated ASL words, signed clearly by native signers. However, the original dataset contains only RGB videos without any annotated… view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Comparison of MPJPE and BLEU scores 5 Future Improvements • Investigate fine-tuning strategies for BART and T5 architectures to enhance their performance on domain￾specific English-to-ASL gloss translation tasks. • Develop ensemble methods that synergistically combine …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages

  1. [1]

    documentation debt

    Jack Bandy and Nicholas Vincent. Addressing "documentation debt" in machine learning research: A retrospective datasheet for bookcorpus, 2021

    work page 2021

  2. [2]

    Sign language recognition, generation, and translation: An interdisciplinary perspective, 2019

    Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudrealt, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian V ogler, and Meredith Ringel Morris. Sign language recognition, generation, and translation: An interdisciplinary perspective, 2019

    work page 2019

  3. [3]

    Neural sign language translation

    Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. Neural sign language translation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7784–7793, 2018

    work page 2018

  4. [4]

    Signnet ii: A transformer- based two-way sign language translation model

    Lipisha Chaudhary, Tejaswini Ananthanarayana, Enjamamul Hoq, and Ifeoma Nwogu. Signnet ii: A transformer- based two-way sign language translation model. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(11):12896–12907, 2023

    work page 2023

  5. [5]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Micheal Zeng, and Furu Wei. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16:1505–1518, 2021

    work page 2021

  6. [6]

    Towards an open platform for machine translation of spoken languages into sign languages

    Rostand Costa, Tiago Araujo, Manuella Aschoff, Vinícius Veríssimo, Renan Andrade, Suanny Vieira, Alexandre Santos, Guido Souza Filho, Marília Soares, and Vitor Hanael. Towards an open platform for machine translation of spoken languages into sign languages. Machine Translation, 33, 12 2019

    work page 2019

  7. [7]

    3d avatar approach for continuous sign movement using speech/text

    Debashis Das Chakladar, Pradeep Kumar, Shubham Mandal, Partha Roy, Masakazu Iwamura, and Byung-Gyu Kim. 3d avatar approach for continuous sign movement using speech/text. Applied Sciences, 11:3439, 04 2021

    work page 2021

  8. [8]

    Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

    2019

Show all 41 references
  1. [9]

    How2sign: A large-scale multimodal dataset for continuous american sign language, 2021

    Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro i Nieto. How2sign: A large-scale multimodal dataset for continuous american sign language, 2021

    2021

  2. [10]

    Building a Swiss German Sign Language avatar with JASigning and evaluating it among the Deaf community , volume 15

    Sarah Ebling and John Glauert. Building a Swiss German Sign Language avatar with JASigning and evaluating it among the Deaf community , volume 15. 05 2015

    2015

  3. [11]

    Rwth-phoenix-weather: A large vocabulary sign language recognition and translation corpus

    Jens Forster, Christoph Schmidt, Thomas Hoyoux, Oscar Koller, Uwe Zelle, Justus Piater, and Hermann Ney. Rwth-phoenix-weather: A large vocabulary sign language recognition and translation corpus. 05 2012

    2012

  4. [12]

    In the wild human pose estimation using explicit 2d features and intermediate 3d representations

    Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Gerard Pons-Moll, and Christian Theobalt. In the wild human pose estimation using explicit 2d features and intermediate 3d representations. pages 10897–10906, 06 2019

    2019

  5. [13]

    Contextnet: Improving convolutional neural networks for automatic speech recognition with global context, 2020

    Wei Han, Zhengdong Zhang, Yu Zhang, Jiahui Yu, Chung-Cheng Chiu, James Qin, Anmol Gulati, Ruoming Pang, and Yonghui Wu. Contextnet: Improving convolutional neural networks for automatic speech recognition with global context, 2020

    2020

  6. [14]

    Deep residual learning for image recognition, 2015

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015

    2015

  7. [15]

    Rtmpose: Real-time multi-person pose estimation based on mmpose, 2023

    Tao Jiang, Peng Lu, Li Zhang, Ningsheng Ma, Rui Han, Chengqi Lyu, Yining Li, and Kai Chen. Rtmpose: Real-time multi-person pose estimation based on mmpose, 2023

    2023

  8. [16]

    Bag of tricks for efficient text classification, 2016

    Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification, 2016

    2016

  9. [17]

    Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Necker- mann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. Marian: Fast neural machine translation in c++, 2018

    2018

  10. [18]

    Deep hand: How to train a cnn on 1 million hand images when your data is continuous and weakly labelled

    Oscar Koller, Hermann Ney, and Richard Bowden. Deep hand: How to train a cnn on 1 million hand images when your data is continuous and weakly labelled. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3793–3802, 2016

    2016

  11. [19]

    A deep learning approach for analyzing video and skeletal features in sign language recognition

    Dimitrios Konstantinidis, Kosmas Dimitropoulos, and Petros Daras. A deep learning approach for analyzing video and skeletal features in sign language recognition. 08 2018

    2018

  12. [20]

    A deep learning approach for analyzing video and skeletal features in sign language recognition

    Dimitrios Konstantinidis, Kosmas Dimitropoulos, and Petros Daras. A deep learning approach for analyzing video and skeletal features in sign language recognition. In 2018 IEEE International Conference on Imaging Systems and Techniques (IST), pages 1–6, 2018. 10

    2018

  13. [21]

    Corchado

    Deep Kothadiya, Chintan Bhatt, Krenil Sapariya, Kevin Patel, Ana-Belén Gil-González, and Juan M. Corchado. Deepsign: Sign language detection and recognition using deep learning. Electronics, 11(11), 2022

    2022

  14. [22]

    Openpifpaf: Composite fields for semantic keypoint detection and spatio-temporal association, 2021

    Sven Kreiss, Lorenzo Bertoni, and Alexandre Alahi. Openpifpaf: Composite fields for semantic keypoint detection and spatio-temporal association, 2021

    2021

  15. [23]

    Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

    Mike Lewis. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019

    1910 arXiv

  16. [24]

    Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison, 2020

    Dongxu Li, Cristian Rodriguez Opazo, Xin Yu, and Hongdong Li. Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison, 2020

    2020

  17. [25]

    Speech enhancement method based on lstm neural network for speech recognition

    Ming Liu, Yujun Wang, Jin Wang, Jing Wang, and Xiang Xie. Speech enhancement method based on lstm neural network for speech recognition. In 2018 14th IEEE International Conference on Signal Processing (ICSP) , pages 245–249, 2018

    2018

  18. [26]

    Bruce T. Lowerre. The harpy speech recognition system. 1976

    1976

  19. [27]

    Mediapipe: A framework for building perception pipelines, 2019

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. Mediapipe: A framework for building perception pipelines, 2019

    2019

  20. [28]

    Jointformer: Single-frame lifting transformer with error prediction and refinement for 3d human pose estimation, 2022

    Sebastian Lutz, Richard Blythman, Koustav Ghosal, Matthew Moynihan, Ciaran Simms, and Aljosa Smolic. Jointformer: Single-frame lifting transformer with error prediction and refinement for 3d human pose estimation, 2022

    2022

  21. [29]

    Efficient estimation of word representations in vector space, 2013

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space, 2013

    2013

  22. [30]

    On feature selection in environmental sound recognition

    Dalibor Mitrovi´c, Matthias Zeppelzauer, and Horst Eidenberger. On feature selection in environmental sound recognition. In 2009 International Symposium ELMAR , pages 201–204, 2009

    2009

  23. [31]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics , ACL ’02, page 311–318, USA, 2002. Association for Computational...

    2002

  24. [32]

    A hybrid of deep cnn and bidirectional lstm for automatic speech recognition

    Vishal Passricha and Rajesh Kumar Aggarwal. A hybrid of deep cnn and bidirectional lstm for automatic speech recognition. Journal of Intelligent Systems, 29:1261 – 1274, 2019

    2019

  25. [33]

    Iterative alignment network for continuous sign language recognition

    Junfu Pu, Wengang Zhou, and Houqiang Li. Iterative alignment network for continuous sign language recognition. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4160–4169, 2019

    2019

  26. [34]

    Robust speech recognition via large-scale weak supervision, 2022

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision, 2022

    2022

  27. [35]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023

    2023

  28. [36]

    K. M. Ravikumar, Balakrishna Reddy, R. Rajagopal, and H. C. Nagaraj. Automatic detection of syllable repetition in read speech for objective assessment of stuttered disfluencies. World Academy of Science, Engineering and Technology, International Journal of Electrical, Compute...

    2008

  29. [37]

    K. M. Ravikumar, R.Rajagopal, and H.C.Nagaraj. An approach for objective assessment of stuttered speech using mfcc features. 2009

    2009

  30. [39]

    Adversarial training for multi-channel sign language production, 2020

    Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. Adversarial training for multi-channel sign language production, 2020

    2020

  31. [40]

    Text2sign: Towards sign language production using neural machine translation and generative adversarial networks

    Stephanie Stoll, Necati Camgoz, Simon Hadfield, and Richard Bowden. Text2sign: Towards sign language production using neural machine translation and generative adversarial networks. International Journal of Computer Vision, 04 2020

    2020

  32. [41]

    There and back again: 3d sign language generation from text using back-translation

    Stephanie Stoll, Armin Mustafa, and Jean-Yves Guillemaut. There and back again: 3d sign language generation from text using back-translation. In 2022 International Conference on 3D Vision (3DV) , pages 187–196, 2022

    2022

  33. [42]

    Unispeech: Unified speech representation learning with labeled and unlabeled data

    Chengyi Wang, Yuehua Wu, Yu Wu, Yao Qian, Ken’ichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, and Xuedong Huang. Unispeech: Unified speech representation learning with labeled and unlabeled data. ArXiv, abs/2101.07597, 2021. 11

    2021 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.