Pith. sign in

REVIEW 4 major objections 5 minor 48 references

Exploring Pose-based Sign Language Translation: Ablation Studies and Attention Insights

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that normalizing pose keypoints to a shoulder-centered 'signing space' more than triples the BLEU-4 score of a T5-based sign language translation model, and that attention analysis suggests adding register tokens would…

desk verdict SignSpace normalization is a genuinely useful, likely robust finding, but the rest of the ablation ranking rests on an unvalidated convergence assumption and should be read as provisional. read the letter →

arxiv 2507.01532 v1 pith:Z7DQ66UN submitted 2025-07-02 cs.CV

Tomas Zelezny , Jakub Straka , Vaclav Javorek , Ondrej Valach , Marek Hruz , Ivan Gruber This is my paper · ORCID
classification cs.CV
keywords signlanguagetranslationposekeypointsT5transformernormalizationdataaugmentationattentionanalysisregistertokensablationstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that for gloss-free sign language translation from pose keypoints, how you preprocess the keypoints can matter as much as the model. Training a modified T5 encoder-decoder on YouTubeASL and evaluating on How2Sign without additional fine-tuning, the authors compare four normalization schemes, three missing-keypoint strategies, and several augmentation protocols. Their central quantitative claim is that 'SignSpace' normalization—a frame-wise normalization to a shoulder-centered signing box with locally normalized hands and face—raises BLEU-4 from 0.73 with no normalization to 2.17, with interpolation and a medium augmentation protocol adding smaller gains. The paper also claims, from attention and integrated-gradient analyses, that the model stores information in non-informative frames, which motivates adding dedicated register tokens.

What carries the argument

The main computational object is SignSpace normalization: body pose keypoints are scaled so that the signing space—defined as a rectangle centered between the shoulders with width and height three times the shoulder distance—maps to the range [-1,1] with its center at [0,0], applied frame by frame, while hand and face keypoints are normalized locally with a 10% border. This scheme keeps global body relations intact while giving each articulator a focused, scale-invariant representation. Two supporting mechanisms carry the ablation results: linear interpolation of missing keypoints across short gaps (two frames or fewer), and a geometric augmentation set (shear, elbow rotation, additive noise) applied uniformly across frames. The attention insight rests on cross-attention spikes on non-informative frames, the phenomenon that motivates the paper's proposal of a register token—a dedicated learned input that would store such non-local information, following the register-buffer idea from vision transformers.

What would settle it

Train the same T5 model on YouTubeASL under each normalization scheme until validation BLEU saturates (or for several times longer than 200,000 iterations) and report the mean and standard deviation across seeds; if SignSpace does not clearly beat the frame-wise unit-box normalization, the headline claim fails. A second, direct test: actually add a register token to the encoder input as the paper proposes; if BLEU-4 does not improve over the no-register baseline, the attention-storage interpretation is not supported.

Watch

Extended reading notes

Core claim

The core discovery is that a linguistically motivated normalization—the 'signing space', a rectangle centered between the shoulders with width and height three times the shoulder distance—outperforms both clip-level and frame-level unit-box normalizations by a wide margin on the How2Sign benchmark. Combining SignSpace normalization, linear interpolation of missing keypoints for gaps up to two frames, and a medium-strength augmentation protocol of shear, elbow rotation, and additive noise yields the best observed BLEU-4 of 2.46, versus 0.73 with no normalization. The paper further shows, through averaged encoder self-attention and integrated gradients, that attention heads specialize in temporal context (past, current, future) and that the decoder consistently and strongly attends to non-informative frames, especially at the end of clips. This leads the authors to hypothesize that the model uses such frames as a memory store, analogous to register tokens in vision transformers, and that adding a dedicated register token could improve translation quality.

Load-bearing premise

The paper's ablations compare models trained for a fixed 200,000 iterations that the authors themselves say are not fully trained, with only the best of three seeds reported; if different preprocessing variants converge at different speeds, the ranking of normalization, interpolation, and augmentation strategies may not hold for fully trained models.

Editorial extensions

If this is right

  • Signing-space normalization is a cheap, model-agnostic preprocessing step that any pose-based sign translation system could adopt, with a threefold BLEU improvement observed here.
  • Interpolating short gaps of missing keypoints instead of filling them with a constant should become the default for pose-based sign language translation.
  • A medium-strength augmentation protocol using shear, elbow rotation, and noise outperforms heavy or no augmentation, while rotation and perspective can degrade translation.
  • The attention analysis suggests that pose-based transformers exploit non-informative frames as information storage; adding register tokens is a concrete and testable next step.
  • The model sometimes produces translations that are more correct than the weak YouTubeASL references, which implies that re-labeling or pseudo-labeling the training captions could yield further gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The SignSpace normalization gains likely transfer to other uncurated sign video datasets beyond YouTubeASL, since it directly targets signer position and scale variability; testing it on other backbones would show whether the effect is model-specific.
  • If the register-token hypothesis is correct, attention spikes on non-informative frames should disappear when a register token is added, providing a clean ablation of the storage mechanism.
  • The success of arm-elbow rotation and noise over global rotation suggests a principle: for sign data, augmentations should preserve the articulatory geometry of the sign rather than the camera geometry; a principled taxonomy of sign-safe augmentations would extend this.
  • The observation of models beating weak references implies that the YouTubeASL caption labels contain systematic errors; an automated pseudo-labeling pipeline that detects and corrects such errors could turn a training-data weakness into a training signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a systematic ablation study of pose-based preprocessing for sign language translation (SLT) with a T5 encoder-decoder transformer. On YouTubeASL training and How2Sign evaluation, it compares four normalization strategies (none, clip-wise, frame-wise, and a proposed SignSpace normalization), three missing-keypoint interpolation settings, and several augmentation protocols, reporting BLEU scores. The main quantitative result is that SignSpace normalization substantially outperforms no normalization (BLEU-4 2.17 vs 0.73). Interpolation and augmentation yield smaller improvements. The paper also provides qualitative analyses of encoder self-attention, cross-attention, and integrated gradients, and suggests that register tokens may improve performance. Code and preprocessed YouTubeASL data are released.

Significance. If the reported rankings are reliable, the study provides practically useful guidance for pose-based SLT preprocessing, particularly the SignSpace normalization, which shows a large gain. The attention analysis, especially the observation of spikes on non-informative frames, is an interesting qualitative contribution. The release of code and preprocessed data is a concrete asset for reproducibility. However, the central quantitative claims are weakened by the absence of training convergence evidence, the use of best-of-three-seed reporting without variance, and test-set-based selection of augmentations. The normalization result is large enough to likely survive these issues, but the interpolation and augmentation conclusions rest on small differences that could be noise.

major comments (4)
  1. [Section 4.1] The quantitative conclusions rest on an unverified assumption that 200,000-iteration models reflect the rankings of fully trained models. The paper explicitly states 'the trained models after 200,000 iterations are not fully trained' and asserts, without supporting evidence, that the comparative performance after this shorter protocol reflects the performance of fully-trained models. With a constant learning rate, different preprocessing schemes can plausibly converge at different rates, and a configuration that appears worse at 200k iterations could overtake another at convergence. The small differences in Tables 2 and 4 (e.g., BLEU-4 2.43 vs 2.17 and 2.46 vs 2.17) could easily reverse. Please provide evidence of ranking stability, such as training the key comparisons to convergence or showing learning curves across multiple checkpoints and seeds that demonstrate the rank order is stable.
  2. [Section 4.4] The augmentation protocol is selected based on test-set performance, which biases the reported gains. Table 3 evaluates individual augmentations on the How2Sign test set, and the text states 'Based on these individual performances, we select those augmentations that positively impact performance to design a final augmentation protocol.' The final protocols in Table 4 are then evaluated on the same test set. This selection procedure inflates the apparent benefit of augmentation. Please use a validation split for selecting augmentations and report test-set results only once, or otherwise account for the multiple-testing problem.
  3. [Section 4.1 and Tables 1-4] Reporting only the best of three seeds without variance or significance testing makes it impossible to assess whether the observed differences are meaningful. The paper states 'we run each experiment with three different seeds and report the best run.' For differences as small as those in Tables 2 and 4, the reported point estimates could easily be within seed noise. Please report the mean and standard deviation (or all per-seed results) for at least the main comparisons, and ideally provide confidence intervals or significance tests. Without this, the abstract's claim that the techniques 'significantly improve' performance is not supported.
  4. [Abstract and Section 5.2] The abstract claims that attention analysis 'suggest[s] that adding a dedicated register token can improve overall model performance,' but no experiment in the paper tests register tokens. Section 5.2 only hypothesizes that register buffers might be helpful, and the conclusion says 'suggesting that register tokens could further enhance SLT performance.' This is a qualitative speculation, not a demonstrated result. Please soften the abstract wording to reflect that this is a hypothesis for future work, or add an actual experiment with a register token.
minor comments (5)
  1. [Title page] The affiliation contains typos: 'Univerzity' should be 'University' and 'Faculty od Applied Sciences' should be 'Faculty of Applied Sciences.'
  2. [Section 4.2] The sentence 'All the proposed normalization results in better performance' has a subject-verb agreement error; it should be 'All the proposed normalization methods result in better performance.'
  3. [Section 4.4] The statement 'the overall performance (majority of the BLEU scores) was improved by the shear, rotate elbow, and noise augmentations' is vague. Please specify which BLEU scores improved and by how much, since Table 3 shows mixed results across BLEU-1 to BLEU-4.
  4. [Section 5.3] The description of integrated gradients would benefit from specifying the baseline and the number of samples used in Figure 8. The text mentions a threshold of 0.3 for visualization but does not state how many samples were averaged or how 'relatively better translated' was operationalized beyond a minimal BLEU-1 score of 10.
  5. [References] Reference [3] is described in the text as work on register buffers, but the citation appears as 'Bondarenko et al.' which is about quantizable transformers; please verify that the citation matches the claim about non-informative tokens.

Circularity Check

1 steps flagged · score 4.0 of 10

Test-set selection of augmentation protocol partially forces the reported augmentation gain; normalization and interpolation ablations remain independent.

  1. fitted input called prediction [Section 4.4 (Augmentations), Tables 3 and 4]
    "First, we assess the contributions of individual augmentations by applying them with a medium-scale value and evaluating the finetuned models. Based on these individual performances, we select those augmentations that positively impact performance to design a final augmentation protocol with three different scales, as described in Section 3.1.4."

    The final augmentation protocol is constructed by selecting the augmentations that improved BLEU on the How2Sign test set (Table 3 identifies shear, rotate elbow, and noise as positive), and Table 4 then reports the BLEU of that same selected protocol on the same test set (medium 2.46 vs none 2.17). Because the selection criterion and the reported metric use the same data and the same metric, the claimed improvement of 'appropriate augmentation' is a selected optimum rather than an independent out-of-sample prediction; the result is statistically forced by the selection step. The normalization and interpolation ablations are defined independently of the test set and are not affected by this circularity.

full rationale

The paper is an empirical ablation study; there is no equation-level derivation whose output is identical to its input. The normalization and interpolation comparisons (Tables 1 and 2) define each preprocessing variant a priori and evaluate it on held-out How2Sign clips, so those results are genuine out-of-sample measurements. The only self-citation (Bohacek and Hruz for the signing space) is attribution of a method that is independently evaluated here, not a load-bearing proof. The one circular step is the augmentation protocol: Section 4.4 selects augmentations by their BLEU on the How2Sign test set and then reports the BLEU of the selected protocol on that same set (Table 4), making the augmentation gain a selected optimum rather than a prediction. The 'not fully trained' caveat in Section 4.1 is a separate reliability threat, not a circularity: the authors state a belief, without supporting evidence, that 200k-iteration rankings match fully-trained rankings; this affects confidence in all tables but is not a reduction to inputs. Overall, the central normalization and interpolation findings are independent, while the augmentation sub-claim is partially forced by test-set selection, giving a partial circularity score of 4.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities and tests no theoretical derivation. Its claims rest on hyperparameter choices and dataset assumptions, listed above. The most consequential assumptions are the sufficiency of the training protocol and the representativeness of the custom split.

free parameters (5)
  • MissingValueConstant = -10
    Used to replace undetected keypoints when no interpolation is applied. Chosen by the authors after a 'large negative number' in YouTubeASL, with no sensitivity analysis. Affects all baseline and interpolation conditions. Section 3.1.3.
  • InterpolationGapThreshold = 2 or 3 frames
    Linear interpolation applied to gaps of length <=2 or <=3 frames. The threshold was chosen based on the observed distribution of missing values (60% within 2 frames, 75% within 3), not from a model. Section 3.1.3.
  • SigningSpaceBoxFactor_Normalization = 3x shoulder distance (vs 4x in extraction)
    The normalization box is set to three times the shoulder distance, while the extraction box uses four times. The 3x value is a hand-tuned design choice that determines what fraction of keypoints fall outside the normalized range. Section 3.1.2.
  • AugmentationProtocolScales = heavy/medium/light with specific probabilities and angles
    Three intensity levels were defined by scaling a default strategy by 0.75 and 0.5. These scales are arbitrary and directly affect the augmentation results. Supplementary Table 1.
  • AugmentationSelection = shear, rotate elbow, noise selected; others dropped
    The final augmentation protocol was constructed by selecting augmentations that improved BLEU on the How2Sign test set. This is a hand-chosen selection made on the evaluation set. Section 4.4.
assumptions (5)
  • domain assumption Pose keypoints extracted by MediaPipe and YOLOv8 are sufficiently accurate and information-complete for SLT.
    The entire pipeline relies on 2D keypoints as the sole input; if hand or finger details are lost or noisy, translation quality is capped regardless of preprocessing. Invoked in Section 3.1.1.
  • domain assumption The custom 90:10 train/validation split of YouTubeASL is representative of the dataset's distribution.
    The paper does not use the official split, which prevents comparison with other published results and may introduce distribution shift. Section 4.1.
  • domain assumption Performance on How2Sign without fine-tuning is a valid measure of generalization ability.
    The model is trained only on YouTubeASL and evaluated on How2Sign; the paper treats this as a robustness test, but domain differences could confound the ablations. Section 4.1.
  • domain assumption BLEU is an appropriate metric for evaluating SLT quality.
    BLEU measures surface n-gram overlap and poorly reflects semantic correctness, yet it is the sole quantitative metric used. Section 4.1.
  • domain assumption Relative performance after 200k iterations (not fully trained) matches relative performance at convergence.
    The paper explicitly states models are not fully trained and asserts without evidence that the ranking is stable. Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Pose-based Sign Language Translation: Ablation Studies and Attention Insights." pith.science (2026). https://pith.science/paper/Z7DQ66UN

@misc{pith2026250701532,
  author       = {Pith},
  title        = {Pith review of: Exploring Pose-based Sign Language Translation: Ablation Studies and Attention Insights},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z7DQ66UN}},
  note         = {Machine review of arXiv:2507.01532}
}
read the original abstract

Sign Language Translation (SLT) has evolved significantly, moving from isolated recognition approaches to complex, continuous gloss-free translation systems. This paper explores the impact of pose-based data preprocessing techniques - normalization, interpolation, and augmentation - on SLT performance. We employ a transformer-based architecture, adapting a modified T5 encoder-decoder model to process pose representations. Through extensive ablation studies on YouTubeASL and How2Sign datasets, we analyze how different preprocessing strategies affect translation accuracy. Our results demonstrate that appropriate normalization, interpolation, and augmentation techniques can significantly improve model robustness and generalization abilities. Additionally, we provide a deep analysis of the model's attentions and reveal interesting behavior suggesting that adding a dedicated register token can improve overall model performance. We publish our code on our GitHub repository, including the preprocessed YouTubeASL data.

Figures

Figures reproduced from arXiv: 2507.01532 by the authors.

Figure 1
Figure 1. Examples of evaluated normalization methods. We com [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Examples of individual augmentations. We show only [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Encoder self-attention averaged over layers per attention [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Cross-Attention averaged over all attention heads in a [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Histogram (a) visualizes Cross-Attention Distribution [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Attribution of Pose Frames to Generated Tokens (top): [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Example video frame sequences where the model has [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 4
Figure 4. Figure 4: In Figure 5, cross-attention matrices are averaged [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 1
Figure 1. Figure 1: Encoder self-attention averaged over layers per attention head. [PITH_FULL_IMAGE:figures/full_fig_p012_1.png]
Figure 2
Figure 2. Figure 2: Encoder self-attention averaged over layers per attention head. [PITH_FULL_IMAGE:figures/full_fig_p013_2.png]
Figure 3
Figure 3. Figure 3: Encoder self-attention averaged over layers per attention head. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png]
Figure 4
Figure 4. Figure 4: Cross-Attention averaged for each layer over all attention heads, showing temporal progression of tokens attending to frames. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Cross-Attention averaged for each attention head over all layers, showing temporal progression of tokens attending to frames. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Cross-Attention averaged for each layer over all attention heads, showing temporal progression of tokens attending to frames. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Cross-Attention averaged for each layer over all attention heads, showing temporal progression of tokens attending to frames. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Histogram (a) visualizes Cross-Attention Distribution over all attention heads and layers, with a long intensity spike sequence in [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 42 canonical work pages

  1. [1]

    SignAttention: On the Interpretability of Transformer Models for Sign Language Translation

    Pedro Alejandro Dal Bianco, Oscar Agust ˜An Stanchi, Fa- cundo Manuel Quiroga, Franco Ronchetti, and Enzo Fer- rante. Signattention: On the interpretability of trans- former models for sign language translation. arXiv preprint arXiv:2410.14506, 2024. 2, 8

    work page Pith review arXiv

  2. [2]

    Sign pose-based trans- former for word-level sign language recognition

    Maty ´aˇs Boh ´aˇcek and Marek Hr ´uz. Sign pose-based trans- former for word-level sign language recognition. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, pages 182–191, 2022. 2, 3

    work page 2022

  3. [3]

    Quantizable transformers: Removing outliers by helping attention heads do nothing

    Y elysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Quantizable transformers: Removing outliers by helping attention heads do nothing. Advances in Neural Information Processing Systems , 36:75067–75096, 2023. 7

    work page 2023

  4. [4]

    Neural sign language trans- lation

    Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Her- mann Ney, and Richard Bowden. Neural sign language trans- lation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 1

    work page 2018

  5. [5]

    Multi-channel transformers for multi- articulatory sign language translation

    Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. Multi-channel transformers for multi- articulatory sign language translation. In Computer Vision– ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pages 301–319. Springer, 2020. 2

    work page 2020

  6. [6]

    Sign language transformers: Joint end-to- end sign language recognition and translation

    Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. Sign language transformers: Joint end-to- end sign language recognition and translation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10023–10033, 2020. 1, 2

    work page 2020

  7. [7]

    A simple multi-modality transfer learning baseline for sign language translation

    Y utong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, and Stephen Lin. A simple multi-modality transfer learning baseline for sign language translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5120–5130, 2022. 2

    work page 2022

  8. [8]

    Vision transformers need registers, 2024

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers, 2024. 7

    work page 2024

Show all 48 references
  1. [9]

    Asl cit- izen: a community-sourced dataset for advancing isolated sign language recognition

    Aashaka Desai, Lauren Berger, Fyodor Minakov, Nessa Mi- lano, Chinmay Singh, Kriston Pumphrey, Richard Ladner, Hal Daum ´e III, Alex X Lu, Naomi Caselli, et al. Asl cit- izen: a community-sourced dataset for advancing isolated sign language recognition. Advances in Neural Info...

    2023

  2. [10]

    How2sign: A large-scale multi- modal dataset for continuous american sign language, 2021

    Amanda Duarte, Shruti Palaskar, Lucas V entura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro i Nieto. How2sign: A large-scale multi- modal dataset for continuous american sign language, 2021. 1, 2

    2021

  3. [11]

    Signllm: Sign languages production large language models

    Sen Fang, Lei Wang, Ce Zheng, Y apeng Tian, and Chen Chen. Signllm: Sign languages production large language models. arXiv preprint arXiv:2405.10718, 2024. 2

    2024 arXiv

  4. [12]

    Llms are good sign language translators, 2024

    Jia Gong, Lin Geng Foo, Yixuan He, Hossein Rahmani, and Jun Liu. Llms are good sign language translators, 2024. 2

    2024

  5. [13]

    Multi-stream keypoint attention network for sign language recognition and translation

    Mo Guan, Y an Wang, Guangkun Ma, Jiarui Liu, and Mingzu Sun. Multi-stream keypoint attention network for sign language recognition and translation. arXiv preprint arXiv:2405.05672, 2024. 2

    2024 arXiv

  6. [14]

    Hierarchical lstm for sign language translation

    Dan Guo, Wengang Zhou, Houqiang Li, and Meng Wang. Hierarchical lstm for sign language translation. In Proceed- ings of the AAAI conference on artificial intelligence , 2018. 1

    2018

  7. [15]

    Signbert: pre-training of hand-model- aware representation for sign language recognition

    Hezhen Hu, Weichao Zhao, Wengang Zhou, Y uechen Wang, and Houqiang Li. Signbert: pre-training of hand-model- aware representation for sign language recognition. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 11087–11096, 2021. 1

    2021

  8. [16]

    Signbert+: Hand-model-aware self-supervised pre- training for sign language understanding

    Hezhen Hu, Weichao Zhao, Wengang Zhou, and Houqiang Li. Signbert+: Hand-model-aware self-supervised pre- training for sign language understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):11221– 11239, 2023. 2

    2023

  9. [17]

    Universal gloss-level representation for gloss-free sign language translation and production

    Eui Jun Hwang, Sukmin Cho, Huije Lee, Y oungwoo Y oon, and Jong C Park. Universal gloss-level representation for gloss-free sign language translation and production. arXiv preprint arXiv:2407.02854, 2024. 2

    2024 arXiv

  10. [18]

    A gloss- free sign language production with discrete representation

    Eui Jun Hwang, Huije Lee, and Jong C Park. A gloss- free sign language production with discrete representation. In 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG), pages 1–6. IEEE, 2024. 2

    2024

  11. [19]

    Ultralytics YOLO, 2023

    Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralytics YOLO, 2023. 3

    2023

  12. [20]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. In European Conference on Computer Vision , pages 206–228. Springer, 2024. 2

    2024

  13. [21]

    Captum: A unified and generic model interpretability library for pytorch

    Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Mel- nikov, Natalia Kliushkina, Carlos Araya, Siqi Y an, et al. Captum: A unified and generic model interpretability library for pytorch. arXiv preprint arXiv:2009.07896, 2020. 7

    2009 arXiv

  14. [22]

    Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison

    Dongxu Li, Cristian Rodriguez, Xin Y u, and Hongdong Li. Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison. In Pro- ceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1459–1469, 2020. 1

    2020

  15. [23]

    Transferring cross-domain knowledge for video sign language recognition

    Dongxu Li, Xin Y u, Chenchen Xu, Lars Petersson, and Hongdong Li. Transferring cross-domain knowledge for video sign language recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6205–6214, 2020. 1

    2020

  16. [24]

    Llava-slt: Visual language tuning for sign language transla- tion, 2024

    Han Liang, Chengyu Huang, Y uecheng Xu, Cheng Tang, Weicai Y e, Juze Zhang, Xin Chen, Jingyi Y u, and Lan Xu. Llava-slt: Visual language tuning for sign language transla- tion, 2024. 2

    2024

  17. [25]

    Multilingual denoising pre-training for neu- ral machine translation

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neu- ral machine translation. Transactions of the Association for Computational Linguistics, 8:726–742, 2020. 2

    2020

  18. [26]

    Mediapipe: A 9 framework for perceiving and processing reality

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris Mc- Clanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo- Ling Chang, Ming Y ong, Juhyun Lee, et al. Mediapipe: A 9 framework for perceiving and processing reality. In Third workshop on computer vision for AR/VR at IEEE compu...

    2019

  19. [27]

    Diffslt: Enhancing diversity in sign language translation via diffusion model, 2024

    JiHwan Moon, Jihoon Park, Jungeun Kim, Jongseong Bae, Hyeongwoo Jeon, and Ha Y oung Kim. Diffslt: Enhancing diversity in sign language translation via diffusion model, 2024. 2

    2024

  20. [28]

    Diffslt: Enhancing diversity in sign language translation via diffusion model

    JiHwan Moon, Jihoon Park, Jungeun Kim, Jongseong Bae, Hyeongwoo Jeon, and Ha Y oung Kim. Diffslt: Enhancing diversity in sign language translation via diffusion model. arXiv preprint arXiv:2411.17248, 2024. 2

    2024 arXiv

  21. [29]

    Im- pact of face swapping and data augmentation on sign lan- guage recognition

    Marina Perea-Trigo, Enrique J L ´opez-Ortiz, Luis M Soria- Morillo, Juan A ´Alvarez-Garc´ıa, and JJ V egas-Olmos. Im- pact of face swapping and data augmentation on sign lan- guage recognition. Universal Access in the Information So- ciety, pages 1–12, 2024. 2

    2024

  22. [30]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Y anqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. 2, 4

    2023

  23. [31]

    Towards privacy-aware sign language translation at scale, 2024

    Phillip Rust, Bowen Shi, Skyler Wang, Necati Cihan Camg¨oz, and Jean Maillard. Towards privacy-aware sign language translation at scale, 2024. 2

    2024

  24. [32]

    Open-domain sign language translation learned from online video

    Bowen Shi, Diane Brentari, Greg Shakhnarovich, and Karen Livescu. Open-domain sign language translation learned from online video. In EMNLP, 2022. 1

    2022

  25. [33]

    Slgtformer: An attention-based approach to sign language recognition

    Neil Song and Y u Xiang. Slgtformer: An attention-based approach to sign language recognition. arXiv preprint arXiv:2212.10746, 2022. 2

    2022 arXiv

  26. [34]

    Y outube-sl-25: A large- scale, open-domain multilingual sign language parallel cor- pus, 2024

    Garrett Tanzer and Biao Zhang. Y outube-sl-25: A large- scale, open-domain multilingual sign language parallel cor- pus, 2024. 1

    2024

  27. [35]

    Y outube- asl: A large-scale, open-domain american sign language- english parallel corpus, 2023

    David Uthus, Garrett Tanzer, and Manfred Georg. Y outube- asl: A large-scale, open-domain american sign language- english parallel corpus, 2023. 1, 2, 3

    2023

  28. [36]

    Sign2gpt: Leveraging large language models for gloss-free sign language translation, 2024

    Ryan Wong, Necati Cihan Camgoz, and Richard Bowden. Sign2gpt: Leveraging large language models for gloss-free sign language translation, 2024. 2

    2024

  29. [37]

    Modelling sign lan- guage with encoder-only transformers and human pose esti- mation keypoint data

    Luke T Woods and Zeeshan A Rana. Modelling sign lan- guage with encoder-only transformers and human pose esti- mation keypoint data. Mathematics, 11(9):2129, 2023. 2

    2023

  30. [38]

    Multilingual sentence- t5: Scalable sentence encoders for multilingual applications

    Chihiro Y ano, Akihiko Fukuchi, Shoko Fukasawa, Hideyuki Tachibana, and Y otaro Watanabe. Multilingual sentence- t5: Scalable sentence encoders for multilingual applications. arXiv preprint arXiv:2403.17528, 2024. 2

    2024 arXiv

  31. [39]

    Improving gloss-free sign language transla- tion by reducing representation density

    Jinhui Y e, Xing Wang, Wenxiang Jiao, Junwei Liang, and Hui Xiong. Improving gloss-free sign language transla- tion by reducing representation density. arXiv preprint arXiv:2405.14312, 2024. 2

    2024 arXiv

  32. [40]

    Gloss attention for gloss-free sign language translation

    Aoxiong Yin, Tianyun Zhong, Li Tang, Weike Jin, Tao Jin, and Zhou Zhao. Gloss attention for gloss-free sign language translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2551–2562, 2023. 2

    2023

  33. [41]

    Sltunet: A simple unified model for sign language translation

    Biao Zhang, Mathias M ¨uller, and Rico Sennrich. Sltunet: A simple unified model for sign language translation. arXiv preprint arXiv:2305.01778, 2023. 2

    2023 arXiv

  34. [42]

    Scaling sign language translation, 2024

    Biao Zhang, Garrett Tanzer, and Orhan Firat. Scaling sign language translation, 2024. 2

    2024

  35. [43]

    Conditional sentence generation and cross-modal reranking for sign language translation

    Jian Zhao, Weizhen Qi, Wengang Zhou, Nan Duan, Ming Zhou, and Houqiang Li. Conditional sentence generation and cross-modal reranking for sign language translation. IEEE Transactions on Multimedia, 24:2662–2672, 2021. 2

    2021

  36. [44]

    Cvt-slr: Contrastive visual-textual transformation for sign language recognition with variational alignment

    Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, and Stan Z Li. Cvt-slr: Contrastive visual-textual transformation for sign language recognition with variational alignment. In Proceedings of the IEEE/CVF conference on computer vision and pattern ...

    2023

  37. [45]

    Gloss-free sign language translation: Improving from visual- language pretraining

    Benjia Zhou, Zhigang Chen, Albert Clap ´es, Jun Wan, Y anyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. Gloss-free sign language translation: Improving from visual- language pretraining. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages ...

    2023

  38. [46]

    Dynamic pseudo label decoding for continuous sign language recogni- tion

    Hao Zhou, Wengang Zhou, and Houqiang Li. Dynamic pseudo label decoding for continuous sign language recogni- tion. In 2019 IEEE International conference on multimedia and expo (ICME), pages 1282–1287. IEEE, 2019. 8

    2019

  39. [47]

    Improving sign language translation with monolingual data by sign back-translation

    Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. Improving sign language translation with monolingual data by sign back-translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 1316–1325, 2021. 2 10 Supplementary ...

    2021

  40. [48]

    The rotate augmentation rotates all keypoints around the center of the bounding box derived from the body pose keypoints

    We use standard geometric augmentations. The rotate augmentation rotates all keypoints around the center of the bounding box derived from the body pose keypoints. Shear is applied along either the x- or y-axis. Perspective transfor- mation is applied to either the top and bott...

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.