Pith. sign in

REVIEW 3 major objections 6 minor 94 references

Bridging the Gap Between Semantics and Reconstruction:Unifying Sign Language Translation and Production

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One autoregressive model can translate sign language to text and generate sign language from text.

desk verdict A genuinely useful unification of sign-language translation and production, with a clean tokenizer split and thorough ablations; the headline SLP gains are real but rest on reimplemented baselines and a self-owned eval, so they need confirmation. read the letter →

arxiv 2608.09045 v1 pith:5T4PPNVD submitted 2026-08-10 cs.CL cs.AIcs.CVcs.MM

Xiao Liu , Shiwei Gan , Yafeng Yin , Jiaxin Yin , Bowen Guo , Yaqi Sun , Zhiwei Jiang , Lei Xie This is my paper · ORCID
classification cs.CLcs.AIcs.CVcs.MM
keywords signlanguagetranslationproductionresidualvectorquantizationautoregressivegenerationdiscretetokenspose-basedsemantic–reconstructiongapunifiedsign–textmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that sign language translation (sign to text) and sign language production (text to sign) can be handled by a single autoregressive model instead of separate task-specific architectures. The proposed Uni-SLTP converts continuous pose sequences into discrete tokens with a semantic-reconstruction guided residual vector quantizer, then casts both directions as conditional next-token prediction in one shared vocabulary. On two public sign language datasets, the model reports better motion accuracy for production than prior task-specific systems while staying competitive with pose-based translation baselines. If correct, the claim matters because it turns two inverse modality-mapping problems into one modeling problem, reducing duplication and allowing the two tasks to share representations.

What carries the argument

The load-bearing mechanism is Semantic-Reconstruction guided Residual Vector Quantization (SR-RVQ), an N-stage residual vector quantizer whose first codebook is supervised as a semantic anchor with gloss CTC prediction and a contrastive pose-text alignment loss, while later codebooks encode residual motion detail under a reconstruction loss. This separation lets one discrete token space be both text-aligned and decodable to accurate pose, and it lets a single autoregressive model treat pose-to-text (using only the semantic stage) and text-to-pose (using all stages) as the same next-token-prediction operation. The unified vocabulary is built by adding stage-aware disjoint token blocks to the pretrained language model's text vocabulary and masking the output space so text decoding and pose decoding each see only their valid subsets.

What would settle it

Train or evaluate the same Uni-SLTP pipeline on a sign dataset where minimal pairs are distinguished only by facial expression or mouthing while the extracted pose sequences are identical, and check whether translation accuracy or production naturalness drops to near chance; if it does, the pose-only token space is the ceiling.

Watch

Extended reading notes

Core claim

Uni-SLTP's central discovery is that a single tokenizer can satisfy both semantic alignment and motion reconstruction if the two objectives are separated across quantization stages instead of merged. The first residual vector quantization stage is trained as a semantic anchor with gloss CTC and text-contrastive losses, while later residual stages absorb reconstruction pressure through a pose reconstruction loss; gradients from reconstruction are blocked from updating the stage-1 tokens. With these tokens placed in a unified vocabulary alongside text tokens, one pretrained mBART sequence-to-sequence model performs sign language translation by predicting text conditioned on the semantic token stream, and sign language production by predicting all stage-wise pose tokens conditioned on text. Experimental comparisons show that the model produces lower hand and body motion error than task-specific production baselines and translation scores on par with pose-only translation baselines.

Load-bearing premise

The model assumes that a 133-dimensional SMPL-X pose sequence, without RGB appearance, carries enough linguistic information for sign language translation; if facial expressions, mouthing, and other non-manual signals are linguistically essential, both translation and production quality are capped.

Editorial extensions

If this is right

  • A single model can serve both translation and production, so sign language systems no longer need separate pipelines for the two directions.
  • Using only the semantic token stream for translation and all streams for production gives a tunable trade-off between translation quality, motion accuracy, and inference cost.
  • Fine-tuning the unified model on a single task improves that task further, indicating that the jointly trained model is a strong shared initialization for task-specific deployment.
  • Within the tested ranges, deeper residual quantization and larger codebooks can improve production motion accuracy without sacrificing translation performance.
  • The same discrete pose token space can be reused by other autoregressive or language-model-based systems, since pose tokens behave exactly like text tokens in the unified vocabulary.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The semantic anchor stream is already supervised by gloss CTC, so extending the same tokenizer to isolated or continuous sign language recognition is a natural untested corollary that would add understanding subtasks to the unified model.
  • Because the model is pose-only, its ceiling is set by what 133 SMPL-X parameters can carry; adding facial expression, mouthing, or other non-manual signals would be a direct extension that could close the gap to RGB-based translation systems.
  • The back-translation metric the paper uses could be turned into a training signal, letting generated poses be translated back into text and used to iteratively refine both directions, a cycle the paper evaluates but does not exploit.
  • If SR-RVQ is trained on a multilingual sign corpus, the shared token space might support cross-lingual translation and production in one model, a testable extension beyond the two datasets reported.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Uni-SLTP, a unified autoregressive framework for sign language translation (SLT) and sign language production (SLP) from pose sequences. The core idea is a Semantic-Reconstruction guided Residual Vector Quantizer (SR-RVQ) that discretizes SMPL-X pose sequences into a semantic first-stage token stream plus residual motion token streams; a frozen SR-RVQ tokenizer then feeds a shared mBART-based sequence model that generates text from semantic pose tokens (SLT) and generates multistage pose tokens from text (SLP). Experiments on Phoenix14T and CSL-Daily report SLT that is competitive with pose-based baselines and SLP that improves over prior methods in back-translation and motion metrics, together with ablations of the tokenizer design, RVQ depth, codebook size, loss weights, and LM backbone.

Significance. If the empirical claims hold, the paper is a useful step for the sign-language and multimedia communities: it demonstrates that two opposite-direction sign-text tasks can share one autoregressive backbone, and SR-RVQ is a sensible mechanism for balancing semantic alignment and motion reconstruction. The ablation suite is extensive (Tables 4-9), and the cross-backbone check in Table 6 supports the claim that the gains come from the representation and training design rather than from the specific LM. The main caveats are that the SLP superiority claim depends on reimplemented baselines and an in-house back-translation evaluator, and that the pose-only setting caps the generality of the framework; these issues need to be addressed before the central empirical claims can be considered fully established.

major comments (3)
  1. [§4.2, Table 2] The SLP motion-accuracy claim rests on comparisons with T2M-GPT* and SOKE*, which are reimplemented by the authors, and no variance estimates or significance tests are reported. For example, on Phoenix14T test the DTW-Body margin over SOKE* is 5.80 vs 6.16 and DTW-Hand is 1.72 vs 1.85; on CSL-Daily the B-T BLEU-4 margin is 11.41 vs 10.31. These margins are small enough that differences in pose extraction, codebook training, metric alignment, or evaluation scripts could change the ranking. The authors should run official released checkpoints where available, including the closely related T2S-GPT [73], which is cited in Section 2 but absent from Table 2, or release their reimplementations and report multiple-seed confidence intervals. Without this, the central claim of superior SLP motion accuracy is not established.
  2. [§4.2, back-translation metric] The back-translation semantic metric is computed with MixSignGraph [15], a translator trained by the authors, and no independent SLT model is used as a cross-check. B-T scores can therefore reflect compatibility between the generated poses and the authors' own pose encoder rather than general linguistic fidelity. Section 4.1 promises BLEU-1/4 for back-translation, but Table 2 reports only BLEU-4; the missing BLEU-1 and the absence of an external evaluator should be addressed before the B-T advantage is claimed.
  3. [§4.1, pose-only assumption] The pose-only input, S ∈ R^{T×133} of SMPL-X parameters, discards facial expressions, mouthing, and part of finger-spelling, which the authors acknowledge as the reason pose-based SLT trails RGB-based systems in Table 1. Because the title and framing claim to unify sign language translation and production in general, the intended scope should be stated explicitly in the abstract and conclusion: the contributions are for pose-based SLT/SLP. The paper should also discuss what would be needed to extend SR-RVQ to appearance-based sign representations, rather than leaving this boundary implicit.
minor comments (6)
  1. [§3.3, Eq. (8)] The serialization order of the multi-stage pose tokens is ambiguous as written: [<sos>, q1,1, ..., qN,1, ..., q1,T', ..., qN,T', <eos>] could mean either stage-major or time-major ordering within each frame block; please define the exact flattening rule formally.
  2. [§1, reference [10]] The introduction cites GPT-2 as reference [10], but [10] is not the GPT-2 paper; please replace it with the correct citation for the GPT-2 model.
  3. [§4.1, evaluation metrics] The evaluation section states that JPE/MPJPE and their DTW counterparts are reported, but Table 2 only shows DTW-PA-JPE; please clarify which metrics are used in the final comparison or add the missing columns.
  4. [Table 3] The comparison with unified human motion methods does not state whether MotionGPT, MotionGPT2, and MG-MotionLLM were run from official checkpoints or reimplemented, nor what adaptation was performed for sign-language data; this should be clarified.
  5. [Table 4] The column alignment in Table 4 is difficult to follow in the typeset version because the check marks and dashes are not cleanly matched to the headers; please reformat the table so each supervision signal is clearly aligned with its column.
  6. [§4.3, Figure 4] The qualitative comparison shows only two examples; given the quantitative claim of finer hand-level detail, additional examples or a small user study would strengthen the presentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: Uni-SLTP is trained and evaluated on external datasets; the only author self-citations appear in evaluation infrastructure and are not load-bearing.

full rationale

Uni-SLTP's derivation chain is self-contained rather than circular. The sign tokenizer SR-RVQ is trained with an explicit multi-term objective (Eq. 6): CTC and contrastive losses on the first quantization stage, reconstruction loss on residual stages. The autoregressive model is trained with teacher-forced likelihoods (Eqs. 7-8) on Phoenix14T and CSL-Daily. Reported SLT and SLP numbers are evaluated on held-out dev/test splits with standard ROUGE/BLEU and DTW-PA-JPE metrics; none of these metrics is the training objective of the model being evaluated, and no equation reduces a predicted quantity to a fitted parameter by construction. The paper explicitly documents design tradeoffs (e.g., Table 5 shows using only Q1 for SLT and all stages for SLP; Table 7 reports loss-weight sensitivity), which is ordinary hyperparameter selection rather than a fitted input renamed as a prediction. The only self-citations of note are [15] (MixSignGraph) and [39] (SignPR), used for the back-translation evaluation protocol and as a translator for the B-T metric. This is shared evaluation infrastructure, not a derivation step: the SLP model does not optimize the B-T score, and the motion-accuracy claim rests on DTW-PA-JPE, which is independent of the authors' translator. Reimplementation of starred baselines is a reproducibility risk, not circularity, because no specific reduction to a fitted value or to a self-citation chain is exhibited. No uniqueness theorem is imported from prior work, and the unified formulation is benchmarked against external methods such as MotionGPT and MotionGPT2. Accordingly, no circular step is established.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the pose representation, gloss availability, and the evaluation pipeline. The main tunable choices are RVQ depth, codebook size, and loss weights, all set by ablation on the same benchmarks. No new physical entities are postulated.

free parameters (5)
  • Number of RVQ stages N = 3
    Depth chosen by ablation on Phoenix14T (Table 8); N=3 gives the best SLP DTW with balanced code usage.
  • Codebook size K per stage = 512
    Chosen by ablation (Table 9); K=512 gives the best overall trade-off between SLT and SLP.
  • Loss weights (lambda_rec, lambda_CTC, lambda_Con) = 1.0, 0.8, 0.2
    Selected from the loss-weight sweep in Table 7; the chosen point maximizes SLT ROUGE among tested settings.
  • Temporal downsampling factor s = 4
    Fixed design choice in the pose encoder; not ablated.
  • InfoNCE temperature tau = not reported
    Used in Eq. 5; the value is not stated, which is a missing detail for reproduction.
assumptions (5)
  • domain assumption SMPL-X pose sequences (133 dims) preserve enough linguistic and motion information for both SLT and SLP.
    Used throughout; pose-only input is why Uni-SLTP trails RGB-based SLT in Table 1. Section 4.1.
  • domain assumption Gloss sequences are available for CTC supervision of the semantic stage.
    The CTC loss in Eq. 6 requires gloss annotations on Phoenix14T and CSL-Daily; gloss-free datasets would break the tokenizer pretraining. Section 3.2.
  • domain assumption A pretrained seq2seq LM (mBART) can be extended to a unified vocabulary without catastrophic forgetting under progressive training.
    Stage 2 fine-tunes the LM on SLP then SLT then joint training; no analysis of forgetting or interference is provided. Section 3.3.
  • standard math In-batch InfoNCE with cosine similarity provides a valid semantic alignment signal between pose and text.
    Eq. 5 is a standard contrastive objective; its adequacy for sign-text alignment is assumed.
  • domain assumption Back-translation with MixSignGraph provides a valid measure of SLP semantic consistency.
    Section 4.2: the B-T evaluator is trained by the authors; no external or human evaluation is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging the Gap Between Semantics and Reconstruction:Unifying Sign Language Translation and Production." pith.science (2026). https://pith.science/paper/5T4PPNVD

@misc{pith2026260809045,
  author       = {Pith},
  title        = {Pith review of: Bridging the Gap Between Semantics and Reconstruction:Unifying Sign Language Translation and Production},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5T4PPNVD}},
  note         = {Machine review of arXiv:2608.09045}
}
read the original abstract

Recent advances in sign language (SL) research have shown a trend toward unifying multiple sign language understanding (SLU) subtasks, such as isolated sign language recognition (ISLR), continuous sign language recognition (CSLR), and sign language translation (SLT), within a single framework, leading to substantial progress. Meanwhile, sign language production (SLP), which generates sign sequences from text, has also attracted growing attention. This naturally raises an important question: can sign language understanding and production be unified within a single framework? Compared with unifying SLU subtasks, this problem is substantially more challenging. Existing SLU tasks largely share the same direction of mapping, namely from sign inputs to linguistic outputs, whereas SLT and SLP lie in opposite directions of sign-text mapping. A unified framework must therefore address two key challenges: (1) bridging the modality gap between continuous sign motions and discrete text tokens through a shared sign tokenizer that supports both linguistic abstraction and motion reconstruction; and (2) learning a single conditional autoregressive model that can take either sign or text as input and generate the corresponding target sequence in the opposite modality. To this end, we propose Uni-SLTP, a unified framework for SLT and SLP with two key components: (1) a shared sign tokenizer that converts sign sequences into discrete tokens and latent representations, capturing both semantic and reconstructive information; and (2) a unified autoregressive generation model that formulates both tasks as conditional sequence generation. Experiments on widely used public datasets show that Uni-SLTP achieves superior motion accuracy for SLP while maintaining competitive SLT performance.

Figures

Figures reproduced from arXiv: 2608.09045 by the authors.

Figure 1
Figure 1. Previous task-specific SLT/SLP models vs Uni-SLTP. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. SR-RVQ tokenizer pretraining. A pose encoder maps sign pose sequences to latent features, which are quantized by [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Uni-SLTP: unified autoregressive modeling for SLT and SLP. SR-RVQ converts pose sequences into discrete pose [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: SLP qualitative results of our Uni-SLTP and baseline method SOKE on Phoenix14T (left) and CSL-Daily (right) datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

94 extracted references · 63 canonical work pages

  1. [73]

    Aoxiong Yin, Haoyuan Li, Kai Shen, Siliang Tang, and Yueting Zhuang. 2024. T2S-GPT: Dynamic vector quantization for autoregressive sign language produc- tion from text. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 3345–3356

    work page 2024

  2. [15]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Lei Xie, Sanglu Lu, and Hongkai Wen

    work page

  3. [1]

    Vasileios Baltatzis, Rolandos Alexandros Potamias, Evangelos Ververas, Guanx- iong Sun, Jiankang Deng, and Stefanos Zafeiriou. 2024. Neural Sign Actors: A diffusion model for 3D sign language production from text. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1985–1995

    2024

  4. [2]

    Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, and Richard Bowden. 2017. Subunets: End-to-end hand shape and continuous sign language recognition. In ICCV. IEEE, 3075–3084

    2017

  5. [3]

    Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. 2018. Neural sign language translation. InProceedings of the IEEE conference on computer vision and pattern recognition. 7784–7793

    2018

  6. [4]

    Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. 2020. Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation. InCVPR. 10023–10033

    2020

  7. [5]

    Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, and Stephen Lin. 2022. A simple multi-modality transfer learning baseline for sign language translation. InCVPR. 5120–5130

    2022

  8. [6]

    Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, and Brian Mak. 2022. Two-stream network for sign language recognition and translation.Advances in Neural Information Processing Systems35 (2022), 17043–17056

    2022

Show all 94 references
  1. [7]

    Ka Leong Cheng, Zhaoyang Yang, Qifeng Chen, and Yu-Wing Tai. 2020. Fully convolutional networks for continuous sign language recognition. InECCV. Springer

    2020

  2. [8]

    Runpeng Cui, Hu Liu, and Changshui Zhang. 2017. Recurrent convolutional neural networks for continuous sign language recognition by staged optimization. InCVPR. 7361–7369

    2017

  3. [9]

    Runpeng Cui, Hu Liu, and Changshui Zhang. 2019. A deep neural framework for continuous sign language recognition by iterative training.TMM21, 7 (2019), 1880–1891

    2019

  4. [10]

    Kawin Ethayarajh. 2019. How contextual are contextualized word represen- tations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of the 2019 conference on empirical methods in natural language pro- cessing and the 9th international joint conference ...

    2019

  5. [11]

    Sen Fang, Chunyu Sui, Xuedong Zhang, and Yapeng Tian. 2023. Signdiff: Learning diffusion models for american sign language production.arXiv e-prints(2023), arXiv–2308

    2023

  6. [12]

    Shiwei Gan, Xiao Liu, Yafeng Yin, Nan Liu, Kuizhuang Liu, Desibieer Tuerdaken, Zhiwei Jiang, Lei Xie, Sanglu Lu, and Hongkai Wen. 2026. Learning effective sign features without text for gloss-free sign language translation. InProceedings of the IEEE/CVF Conference on Computer ...

    2026

  7. [13]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Hongkai Wen, Lei Xie, and Sanglu Lu

  8. [14]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Kang Xia, Lei Xie, and Sanglu Lu. 2023. Contrastive Learning for Sign Language Recognition and Translation.. InIJCAI, Vol. 23. 763–772

    2023

  9. [16]

    Jia Gong, Lin Geng Foo, Yixuan He, Hossein Rahmani, and Jun Liu. 2024. Llms are good sign language translators. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18362–18372

    2024

  10. [17]

    Shester Gueuwou, Xiaodan Du, Greg Shakhnarovich, and Karen Livescu. 2025. Signmusketeers: An efficient multi-stream approach for sign language translation at scale.Findings of the Association for Computational Linguistics: ACL 2025(2025)

    2025

  11. [18]

    Shester Gueuwou, Xiaodan Du, Greg Shakhnarovich, Karen Livescu, and Alexan- der H. Liu. 2025. SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction. InProceedings of the 63rd An- nual Meeting of the Association for Computational Lin...

    2025

  12. [19]

    Dan Guo, Wengang Zhou, Houqiang Li, and Meng Wang. 2018. Hierarchical lstm for sign language translation. InAAAI, Vol. 32

    2018

  13. [20]

    Jianyuan Guo, Peike Li, and Trevor Cohn. 2025. Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation.arXiv preprint arXiv:2505.15438(2025)

    2025 arXiv

  14. [21]

    Aiming Hao, Yuecong Min, and Xilin Chen. 2021. Self-Mutual Distillation Learn- ing for Continuous Sign Language Recognition. InICCV. 11303–11312

    2021

  15. [22]

    Hezhen Hu, Weichao Zhao, Wengang Zhou, and Houqiang Li. 2023. SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understand- ing.IEEE Transactions on Pattern Analysis and Machine Intelligence(2023)

    2023

  16. [23]

    Hezhen Hu, Wengang Zhou, and Houqiang Li. 2021. Hand-Model-Aware Sign Language Recognition. InProceedings of the AAAI Conference on Artificial Intelli- gence, Vol. 35. 1558–1566

    2021

  17. [24]

    Lianyu Hu, Liqing Gao, Zekang Liu, and Wei Feng. 2022. Temporal lift pooling for continuous sign language recognition. InEuropean conference on computer vision. Springer, 511–527

    2022

  18. [25]

    Lianyu Hu, Liqing Gao, Zekang Liu, and Wei Feng. 2023. Continuous Sign Language Recognition with Correlation Network. InCVPR. 2529–2539

    2023

  19. [26]

    Lianyu Hu, Liqing Gao, Zekang Liu, and Wei Feng. 2023. Self-emphasizing network for continuous sign language recognition. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 854–862

    2023

  20. [27]

    Jie Huang, Wengang Zhou, Qilin Zhang, Houqiang Li, and Weiping Li. 2018. Video-based sign language recognition without temporal segmentation. InAAAI

    2018

  21. [28]

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. 2023. Mo- tiongpt: Human motion as a foreign language.Advances in Neural Information Processing Systems36 (2023), 20067–20079

    2023

  22. [29]

    Tao Jiang, Necati Cihan Camgoz, and Richard Bowden. 2021. Skeletor: Skeletal transformers for robust body-pose estimation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3394–3402

    2021

  23. [30]

    Peiqi Jiao, Yuecong Min, and Xilin Chen. 2024. Visual alignment pre-training for sign language translation. InEuropean Conference on Computer Vision. Springer, 349–367

    2024

  24. [31]

    Peiqi Jiao, Yuecong Min, Yanan Li, Xiaotao Wang, Lei Lei, and Xilin Chen. 2023. CoSign: Exploring Co-occurrence Signals in Skeleton-based Continuous Sign Language Recognition. InICCV. 20676–20686

    2023

  25. [32]

    Oscar Koller, Cihan Camgoz, Hermann Ney, and Richard Bowden. 2019. Weakly supervised learning with multi-stream CNN-LSTM-HMMs to discover sequential parallelism in sign language videos.TPAMI(2019)

    2019

  26. [33]

    Oscar Koller, Sepehr Zargaran, and Hermann Ney. 2017. Re-sign: Re-aligned end-to-end sequence modelling with deep recurrent CNN-HMMs. InCVPR. 4297– 4305

    2017

  27. [34]

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. 2022. Autoregressive image generation using residual quantization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11523–11532

    2022

  28. [35]

    Dongxu Li, Xin Yu, Chenchen Xu, Lars Petersson, and Hongdong Li. 2020. Trans- ferring cross-domain knowledge for video sign language recognition. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6205–6214

    2020

  29. [36]

    Zecheng Li, Wengang Zhou, Weichao Zhao, Kepeng Wu, Hezhen Hu, and Houqiang Li. 2025. Uni-sign: Toward unified sign language understanding at scale.arXiv preprint arXiv:2501.15187(2025)

    2025 arXiv

  30. [37]

    Han Liang, Chengyu Huang, Yuecheng Xu, Cheng Tang, Weicai Ye, Juze Zhang, Xin Chen, Jingyi Yu, and Lan Xu. 2024. LLaVA-SLT: Visual Language Tuning for Sign Language Translation.arXiv preprint arXiv:2412.16524(2024)

    2024 arXiv

  31. [38]

    Chin-Yew Lin and Franz Josef Och. 2004. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. InProceedings of the 42nd annual meeting of the association for computational linguistics (ACL-04). 605–612

    2004

  32. [39]

    Xiao Liu, Shiwei Gan, Yafeng Yin, Bowen Guo, Zhiwei Jiang, Shunmei Meng, Lei Xie, and Sanglu Lu. 2026. SignPR: A Progressive Vector-Quantized Diffu- sion Framework for Sign Language Production. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2026

  33. [40]

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvinine- jad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation.Transactions of the Association for Computational Linguistics8 (2020), 726–742

    2020

  34. [41]

    Jian Ma, Wenguan Wang, Yi Yang, and Feng Zheng. 2024. Ms2sl: multi- modal spoken data-driven continuous sign language production.arXiv preprint arXiv:2407.12842(2024)

    2024 arXiv

  35. [42]

    Yuecong Min, Aiming Hao, Xiujuan Chai, and Xilin Chen. 2021. Visual alignment constraint for continuous sign language recognition. InICCV. 11542–11551

    2021

  36. [43]

    Zhe Niu and Brian Mak. 2020. Stochastic fine-grained labeling of multi-state sign glosses for continuous sign language recognition. InECCV. Springer, 172–186

    2020

  37. [44]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. InACL. 311–318

    2002

  38. [45]

    Maria Parelli, Katerina Papadimitriou, Gerasimos Potamianos, Georgios Pavlakos, and Petros Maragos. 2022. Spatio-temporal graph convolutional networks for continuous sign language recognition. InICASSP. IEEE. MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Xiao Liu et al

    2022

  39. [46]

    Junfu Pu, Wengang Zhou, Hezhen Hu, and Houqiang Li. 2020. Boosting Con- tinuous Sign Language Recognition via Cross Modality Augmentation. InMM. 1497–1505

    2020

  40. [47]

    Junfu Pu, Wengang Zhou, and Houqiang Li. 2018. Dilated convolutional network with iterative optimization for continuous sign language recognition.. InIJCAI, Vol. 3. 7

    2018

  41. [48]

    Fan Qi, Yu Duan, Huaiwen Zhang, and Changsheng Xu. 2024. Signgen: End-to- end sign language video generation with latent diffusion. InEuropean Conference on Computer Vision. Springer, 252–270

    2024

  42. [49]

    Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu. 2025. Tokenflow: Unified image tokenizer for multimodal understanding and generation. InProceedings of the Computer Vision and Pattern Recognition Conference. 2545–2555

    2025

  43. [50]

    Razieh Rastgoo, Kourosh Kiani, Sergio Escalera, and Mohammad Sabokrou. 2021. Sign language production: A review. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3451–3461

    2021

  44. [51]

    Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. 2020. Adver- sarial training for multi-channel sign language production.arXiv preprint arXiv:2008.12405(2020)

    2020 arXiv

  45. [52]

    Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. 2020. Progressive transformers for end-to-end sign language production. InEuropean Conference on Computer Vision. Springer, 687–705

    2020

  46. [53]

    Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. 2021. Continuous 3d multi-channel sign language production via progressive transformers and mixture density networks.International journal of computer vision129, 7 (2021), 2113–2135

    2021

  47. [54]

    Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. 2021. Mixed signals: Sign language production via a mixture of motion primitives. InProceedings of the IEEE/CVF International Conference on Computer Vision. 1919–1929

    2021

  48. [55]

    Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. 2022. Signing at scale: Learning to co-articulate signs for large-scale photo-realistic sign language production. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5141–5151

    2022

  49. [56]

    Stephanie Stoll, Necati Cihan Camgöz, Simon Hadfield, and Richard Bowden

  50. [57]

    Stephanie Stoll, Armin Mustafa, and Jean-Yves Guillemaut. 2022. There and back again: 3d sign language generation from text using back-translation. In2022 International Conference on 3D Vision (3DV). IEEE, 187–196

    2022

  51. [58]

    Shengeng Tang, Dan Guo, Richang Hong, and Meng Wang. 2021. Graph-based multimodal sequential embedding for sign language translation.TMM(2021)

    2021

  52. [59]

    Shengeng Tang, Jiayi He, Dan Guo, Yanyan Wei, Feng Li, and Richang Hong

  53. [60]

    Shengeng Tang, Feng Xue, Jingjing Wu, Shuo Wang, and Richang Hong. 2025. Gloss-driven conditional diffusion models for sign language production.ACM Transactions on Multimedia Computing, Communications and Applications21, 4 (2025), 1–17

    2025

  54. [61]

    Gemma Team. 2025. Gemma 3. (2025). https://goo.gle/Gemma3Report

    2025

  55. [62]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    2023 arXiv

  56. [63]

    In Proceedings of the AAAI Conference on Artificial Intelligence, Vol

    Sign-idd: Iconicity disentangled diffusion for sign language production. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 7266–7274

  57. [64]

    Harry Walsh, Abolfazl Ravanshad, Mariam Rahmani, and Richard Bowden. 2024. A data-driven representation for sign language production. In2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG). IEEE, 1–10

    2024

  58. [65]

    Yuan Wang, Di Huang, Yaqi Zhang, Wanli Ouyang, Jile Jiao, Xuetao Feng, Yan Zhou, Pengfei Wan, Shixiang Tang, and Dan Xu. 2024. Motiongpt-2: A general- purpose motion-language model for motion generation and understanding.arXiv preprint arXiv:2410.21747(2024)

    2024 arXiv

  59. [66]

    Chengcheng Wei, Jian Zhao, Wengang Zhou, and Houqiang Li. 2020. Semantic Boundary Detection with Reinforcement Learning for Continuous Sign Language Recognition.TCSVT31, 3 (2020), 1138–1149

    2020

  60. [67]

    Andreas Voskou, Konstantinos P Panousis, Dimitrios Kosmopoulos, Dimitris N Metaxas, and Sotirios Chatzis. 2021. Stochastic transformer networks with linear competing units: Application to end-to-end sl translation. InProceedings of the IEEE/CVF International Conference on Comp...

    2021

  61. [68]

    Bizhu Wu, Jinheng Xie, Keming Shen, Zhe Kong, Jianfeng Ren, Ruibin Bai, Rong Qu, and Linlin Shen. 2025. Mg-motionllm: A unified framework for motion comprehension and generation across multiple granularities. InProceedings of the Computer Vision and Pattern Recognition Confere...

    2025

  62. [69]

    Pan Xie, Qipeng Zhang, Peng Taiying, Hao Tang, Yao Du, and Zexian Li. 2024. G2p-ddm: Generating sign pose sequence from gloss sequence with discrete diffusion model. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 6234–6242

    2024

  63. [70]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. InProceedings of the 2021 conference of the North American chapter of the association ...

    2021

  64. [71]

    Fangyun Wei and Yutong Chen. 2023. Improving continuous sign language recognition with cross-lingual signs. InProceedings of the IEEE/CVF International Conference on Computer Vision. 23612–23621

    2023

  65. [72]

    Jinhui Ye, Xing Wang, Wenxiang Jiao, Junwei Liang, and Hui Xiong. 2024. Improv- ing Gloss-free Sign Language Translation by Reducing Representation Density. InNeurIPS

    2024

  66. [74]

    Aoxiong Yin, Tianyun Zhong, Li Tang, Weike Jin, Tao Jin, and Zhou Zhao. 2023. Gloss attention for gloss-free sign language translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2551–2562

    2023

  67. [75]

    Zhaoyang Yang, Zhenmei Shi, Xiaoyong Shen, and Yu-Wing Tai. 2019. SF-Net: Structured Feature Network for Continuous Sign Language Recognition.arXiv preprint arXiv:1908.01341(2019)

    2019 arXiv

  68. [76]

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. 2023. Generating human motion from textual descriptions with discrete representations. InProceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    2023

  69. [77]

    Zhihao Zhang, Junfu Pu, Liansheng Zhuang, Wengang Zhou, and Houqiang Li

  70. [78]

    Weichao Zhao, Hezhen Hu, Wengang Zhou, Jiaxin Shi, and Houqiang Li. 2023. BEST: BERT pre-training for sign language recognition with coupling tokeniza- tion. InProceedings of the AAAI conference on artificial intelligence, Vol. 37. 3597– 3605

    2023

  71. [79]

    Huaiwen Zhang, Zihang Guo, Yang Yang, Xin Liu, and De Hu. 2023. C2st: Cross-modal contextualized sequence transduction for continuous sign language recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision. 21053–21062

    2023

  72. [80]

    Hao Zhou, Wengang Zhou, and Houqiang Li. 2019. Dynamic pseudo label decoding for continuous sign language recognition. InICME. IEEE, 1282–1287

    2019

  73. [81]

    Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. 2021. Improv- ing Sign Language Translation with Monolingual Data by Sign Back-Translation. InCVPR. 1316–1325

    2021

  74. [82]

    Hao Zhou, Wengang Zhou, Yun Zhou, and Houqiang Li. 2020. Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition.. InAAAI

    2020

  75. [83]

    Hao Zhou, Wengang Zhou, Yun Zhou, and Houqiang Li. 2021. Spatial-temporal multi-cue network for sign language recognition and translation.TMM(2021)

    2021

  76. [84]

    Benjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. 2023. Gloss-free sign language translation: Improving from visual-language pretraining. InProceedings of the IEEE/CVF International Conference on Computer Vision. 20871–20881

    2023

  77. [85]

    Qidan Zhu, Jing Li, Fei Yuan, and Quan Gan. 2024. Multiscale temporal network for continuous sign language recognition.Journal of Electronic Imaging33, 2 (2024), 023059–023059

    2024

  78. [86]

    Ronglai Zuo and Brian Mak. 2022. C2SLR: Consistency-Enhanced Continuous Sign Language Recognition. InCVPR. 5131–5140

    2022

  79. [87]

    Ronglai Zuo, Rolandos Alexandros Potamias, Evangelos Ververas, Jiankang Deng, and Stefanos Zafeiriou. 2025. Signs as Tokens: A retrieval-enhanced multilingual sign language generator. InProceedings of the IEEE/CVF International Conference on Computer Vision. 23806–23816

    2025

  80. [88]

    Ronglai Zuo, Fangyun Wei, Zenggui Chen, Brian Mak, Jiaolong Yang, and Xin Tong. 2024. A simple baseline for spoken language to sign language translation with 3d avatars. InEuropean Conference on Computer Vision. Springer, 36–54

    2024

  81. [89]

    Wengang Zhou, Weichao Zhao, Hezhen Hu, Zecheng Li, and Houqiang Li. 2025. Scaling up multimodal pre-training for sign language understanding.IEEE Trans- actions on Pattern Analysis and Machine Intelligence(2025)

    2025

  82. [94]

    Ronglai Zuo, Fangyun Wei, and Brian Mak. 2023. Natural language-assisted sign language recognition. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 14890–14900

    2023

  83. [2018]

    InProceedings of the 29th British Machine Vision Conference (BMVC 2018)

    Sign language production using neural machine translation and generative adversarial networks. InProceedings of the 29th British Machine Vision Conference (BMVC 2018). British Machine Vision Association

    2018

  84. [2019]

    Continuous sign language recognition via reinforcement learning. InICIP. IEEE, 285–289

  85. [2024]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Signgraph: A sign sequence is worth graphs of nodes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13470–13479

  86. [2025]

    InThe Thirty-ninth Annual Conference on Neural Information Processing Systems

    MixSignGraph: A Sign Sequence is Worth Mixed Graphs of Nodes. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.