Pith. sign in

REVIEW 3 major objections 4 minor 58 references

SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Visual-first training lifts gloss-free sign translation to 26.74

desk verdict Solid empirical recipe for gloss-free sign language translation with LLMs, but the How2Sign numbers contradict each other and the core pretraining mechanism is untested, so the paper needs revision rather than rejection. read the letter →

arxiv 2608.09006 v1 pith:65QZHW3Z submitted 2026-08-10 cs.CV cs.AI

Shiwei Gan , Xiao Liu , Yafeng Yin , Zhiwei Jiang , Bowen Guo , Lie Xie , Sanglu Lu , Hongkai Wen This is my paper · ORCID
classification cs.CVcs.AI
keywords signlanguagetranslationgloss-freelargemodelspseudo-glosssupervisionCTCpretrainingvisual-prioritizeddistillationknowledgemultimodalLLMfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a large language model can translate sign-language video into text without any gloss annotations, provided two specific problems are fixed: the mismatch between continuous video features and discrete text tokens, and the tendency of autoregressive models to lean on text tokens once both modalities are concatenated. The proposed SignLlama first pretrains its visual encoder with CTC supervision on pseudo-glosses—sequences built from the target sentence by lemmatization, removal of function words, and random dropping—so video features acquire a text-like, discrete organization. It then fine-tunes the language model with a visual-prioritized distillation in which a visual-only prediction path (text masked) is trained to imitate a standard visual-textual path, forcing the model to rely on the video rather than on already-generated text. On four public benchmarks the paper reports BLEU-4 scores of 26.74 on Phoenix14T, 24.76 on CSL-Daily, 9.89 on How2Sign, and 15.16 on OpenASL, with no external sign-language datasets and no extra modalities. If the claims hold, gloss-free translation stops being the weak sibling of gloss-supervised translation, and the main bottleneck moves from annotation cost to the fidelity of automatically generated pseudo-glosses.

What carries the argument

The two carrying mechanisms are FPG-CTC and VPD. FPG-CTC is a pretraining scheme that converts the target sentence into a pseudo-gloss sequence—after lemmatization, removing prepositions and conjunctions, and randomly dropping a fixed fraction (0.2 in the final model) of words—and uses connectionist temporal classification loss to drive the visual encoder toward text-like discrete representations. VPD is a fine-tuning objective that runs two forward passes through the LLM: a teacher path with visual and textual tokens, and a student path with textual tokens masked, with cross-entropy on both paths plus a KL-divergence distillation term (weight $\lambda=5$) pushing the student's distribution toward the teacher's; only the visual-only path is used at inference. Mechanism-wise, FPG-CTC manufactures the alignment signal that gloss labels normally provide, and VPD converts a text-conditioned language model into a video-conditioned one by making the text-free path the operative path.

What would settle it

Retrain on Phoenix14T with the pseudo-gloss token order shuffled before FPG-CTC pretraining, keeping everything else fixed; if BLEU-4 stays near 26.74 instead of falling toward the 5.46 no-pretraining baseline, then the order and content fidelity of pseudo-glosses is not what carries the gain. A human check of whether the generated pseudo-glosses correspond to the signs actually performed in a random sample of videos would settle the same question directly.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the two obstacles to LLM-based gloss-free sign language translation are separable and each has a simple fix. FPG-CTC closes the visual-textual distribution gap: filtered pseudo-glosses generated from target text supervise the visual backbone through CTC loss, producing visual features that behave like discrete tokens and cluster near the text embedding distribution. VPD closes the visual-deprioritization gap: by distilling logits from the visual-textual path into a visual-only path, the model learns to generate the full target sentence from video alone, and attention maps shift from previous text tokens to visual tokens. The combination is what the paper credits for lifting Phoenix14T BLEU-4 from a 5.46 direct-fine-tuning baseline to 26.74, with consistent gains across four datasets; the authors also report that scaling the language model from 1B to 13B gives only marginal improvements at this range, suggesting the visual alignment stage carries most of the benefit.

Load-bearing premise

Everything rests on the assumption that the pseudo-gloss sequences—made by lemmatizing the target sentence, deleting function words, and randomly dropping one word in five—match, in order, what the signer actually performs; if that correspondence is wrong, the CTC pretraining aligns the video encoder to text-derived labels rather than to signed content, and the reported gains would not hold.

Editorial extensions

If this is right

  • Gloss-free translation can match or exceed many gloss-supervised baselines without any gloss annotation step, so gloss labels may not be necessary for high-quality sign language translation.
  • Text-only pseudo-gloss generation plus CTC is enough to discretize sign video features; the reported t-SNE visualizations show the visual features becoming more discriminative and text-like after FPG-CTC pretraining.
  • Visual-prioritized distillation reduces reliance on previous text tokens and addresses exposure bias; the reported attention maps show the model allocating more weight to visual tokens during generation.
  • Larger Llama sizes from 1B to 13B give only marginal gains in the paper's experiments, implying the bottleneck for this task range is visual alignment rather than raw language-model capacity.
  • The full pipeline needs only RGB video and paired text, with no external sign-language datasets or extra sensor modalities, which lowers the data requirements for deploying LLM-based sign translation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's experiments, the VPD recipe—distill a text-conditioned teacher into a vision-only student—is a general corrective for any multimodal LLM in which one modality dominates, since the gradient argument in Section 4 assumes only an autoregressive text-biased model.
  • The rule-based pseudo-gloss generator could be replaced by any text-derived sequence that preserves temporal order; on other action-to-language tasks without glosses, the same CTC pretraining would apply as an untested extension.
  • Not tested by the paper: the optimal random-drop rate probably tracks how much of the target sentence is left implicit in the signing, so datasets with more verbose text may need a higher drop rate than 0.2.
  • The reported 1B-to-13B plateau is a hint, not a proof, that visual-encoder quality and pseudo-gloss fidelity dominate LLM capacity in this task range; holding the encoder fixed and scaling only the LLM past 13B would test it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SignLlama, a gloss-free sign language translation (GFSLT) model that couples a PoolFormer-based visual encoder with Llama 3.2 1B (or larger variants). Two ingredients are introduced: Filtered Pseudo-Gloss CTC Pretraining (FPG-CTC), which derives pseudo-gloss sequences from the target text via lemmatization, stopword removal, and 20% random dropping and then uses them as CTC supervision targets for the visual encoder; and Visual-Prioritized Distillation (VPD), a training scheme that runs a visual-textual prediction path and a visual-only prediction path, distilling the former into the latter to push the model to rely on visual features. Experiments on Phoenix14T, CSL-Daily, How2Sign, and OpenASL report BLEU-4 scores of 26.74, 24.76, 9.89 (test in Table 9; the text of Section 8 says 14.47, referring to the value in Appendix D Table 14), and 15.16 respectively. The authors claim competitive or state-of-the-art performance without using external sign language datasets or extra modalities.

Significance. If the proposed mechanisms work as described, the paper offers a simple and reproducible recipe for adapting LLMs to gloss-free sign language translation, an important and timely problem. The reported gains over strong baselines (e.g., a BLEU-4 jump from 5.46 to 26.74 on Phoenix14T when FPG-CTC is added) are substantial and could motivate further work on pseudo-gloss supervision and visual-prioritized training for multimodal LLMs. The paper also releases pretrained models across scales, which is a tangible community contribution. However, the significance hinges almost entirely on whether FPG-CTC actually aligns the visual encoder to the signed content in the video; if the pseudo-gloss order does not match sign order, the large empirical gains may arise from an unintended inductive bias rather than the claimed alignment mechanism.

major comments (3)
  1. [Section 3.1, Eq. (3); Appendix D, Table 14] The central premise of FPG-CTC is that the filtered target-text token sequence g_p appears in the video in the same temporal order as the signed content. CTC is alignment-based and order-sensitive: it can insert blanks and repetitions but cannot reorder labels. The paper provides no direct evidence that the lemmatized, stopword-filtered, randomly dropped text sequence actually corresponds to the order of signs in the video. For Phoenix14T and CSL-Daily, gold gloss annotations are available, so a quantitative comparison between pseudo-gloss order and gold-gloss order is feasible and should be reported (e.g., order-aware edit distance, or a per-sequence alignment error). The paper's own CSLR results in Table 14 show WERs of about 78% on Phoenix14T and 70% on CSL-Daily, with deletion rates of 73–75%; while high deletion rates do not directly prove order mismatch, they indicate that the pseudo-glosses are weak recognition targets, and the paper does not disentangle deletion from reordering. Without such evidence, the mechanism for the large FPG-CTC gain (5.46 to 26.74 BLEU-4 on Phoenix14T) is unsupported. This is a load-bearing concern because VPD and all comparison tables depend on the visual encoder produced by FPG-CTC.
  2. [Section 3.2, Eq. (5), Figure 1(b)] The 'Visual-Only Mask' used in the VPD visual-only prediction path is not specified precisely. The paper says 'the textual inputs are masked out using a Visual-Only Mask' but does not state whether text token embeddings are replaced by a special mask embedding, whether an attention mask is applied, or how the model is prevented from seeing previous ground-truth tokens when generating the full sequence from visual input alone. This matters because VPD's whole purpose is to create a path that relies exclusively on visual features; the exact masking mechanism determines what information is available at each position. Without a precise specification or pseudo-code, the method is not reproducible.
  3. [Section 6, Tables 1–3; Appendix C, Tables 12–13] Ablation results are reported as single runs without error bars, and hyperparameters λ (VPD distillation weight) and the FPG-CTC drop rate are tuned on the Phoenix14T validation set and then fixed across all other datasets. Several ablation comparisons differ by less than 1 BLEU-4 point (e.g., Table 2: L_vt-only 23.47 vs L_v-only 23.25; Table 4: Llama2 7B 26.26 vs Llama3.2 1B 26.74), so run-to-run variation could change the conclusions about which components are responsible for the improvement. The authors should report mean and standard deviation over at least three seeds for the main ablations and should justify the transfer of hyperparameters to CSL-Daily, How2Sign, and OpenASL, whose data distributions and sign languages differ substantially from Phoenix14T.
minor comments (4)
  1. [Section 8, 'Evaluation on Phoenix14T Dataset'] The text says 'improving the BLUE4 score' — 'BLUE4' should be 'BLEU-4'.
  2. [Appendix A, Table 11] The table entry for Phoenix14T 'PG Voc in FPG-CTC' reads '30,01', which appears to be a typo or formatting error; please verify and fix all numeric entries in this table.
  3. [Section 3.2, Eq. (5)] The distillation term L_KD is defined as a KL divergence without any temperature parameter. If the standard softened knowledge distillation is intended, the temperature should be stated; otherwise, the formulation should clarify why no temperature is used.
  4. [Section 2, Related Work] The sentence 'There are also some works focusing on designing pseudo gloss [9, 15]' cites references [9] and [15]; reference [9] (SignDINO) is a concurrent/related work from the same group and does not appear to use pseudo glosses, and [15] (MixSignGraph) is a graph-based approach. Please re-check the citation mapping.

Circularity Check

1 steps flagged · score 2.0 of 10

Central BLEU results are empirically measured and non-circular; only the Appendix D pseudo-gloss WER argument is self-referential.

  1. fitted input called prediction [Appendix D, 'Ablation Study on Proposed Module', in the paragraph reporting the CSLR performance based on pseudo-gloss sequences (Table 14).]
    "Additionally, we report the CSLR performance based on the pseudo-gloss sequences generated by FPG-CTC (right side of Table 14). Although the pseudo-gloss labels are not ground truth and the resulting WERs are suboptimal, they still demonstrate a reasonable level of recognition performance, suggesting that the pseudo sequences preserve coherent structure and temporal alignment."

    The WERs are computed against the same pseudo-gloss sequences g_p that Section 3.1 generates from the target text and uses as CTC training targets in Eq. (3). Since R(f) is optimized to match g_p, agreement with g_p measures fit to text-derived training labels, not independent confirmation that g_p follows the temporal order of the signed video. The Appendix's conclusion that 'the pseudo sequences preserve coherent structure and temporal alignment' therefore reduces to a consistency check on the model's own training labels and cannot validate the order-faithfulness premise on which FPG-CTC's benefit depends.

full rationale

The headline claim is an empirical benchmark comparison: BLEU-4, ROUGE, and BLEURT are measured on held-out test splits against standard references, so the central result does not reduce to the training labels. FPG-CTC constructs pseudo-glosses from target text for visual-encoder pretraining; this is weak/self-supervised label construction, not a derivation in which the predicted quantity equals an input. VPD is a two-branch CE-plus-KL training objective (Eq. 5), not a claim that X is defined in terms of Y. The paper contains multiple self-citations (SignGraph [11], MixSignGraph [15], SignDINO [9]) and cites [15,17] for CTC/pseudo-gloss inspiration, but none is load-bearing or used to import a uniqueness result. The Limitations section candidly states that pseudo-gloss generation is rule-based and Appendix D's high WER and deletion rates are correctness risks about label fidelity, not additional circularity. The one self-referential step is the Appendix D WER argument; it is auxiliary rather than central, hence the score of 2 rather than higher.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method rests on two empirical training tricks rather than on new physics or math. The main free parameters are the distillation weight and the pseudo-gloss drop rate, both tuned on a single dataset. The assumptions are standard for supervised sequence-to-sequence learning: available target text, valid CTC alignment, and distillation transferring knowledge. No new entities are introduced.

free parameters (2)
  • lambda (VPD distillation weight) = 5
    Tuned on Phoenix14T validation via Figure 2/Table 12; peak BLEU-4 at lambda=5, stable to lambda=10.
  • drop rate in FPG-CTC pseudo-gloss generation = 0.2
    Tuned on Phoenix14T via Figure 2/Table 13; best at 0.2, degrades above 0.3.
assumptions (4)
  • domain assumption CTC loss can align video frame features to pseudo-gloss sequences generated from text.
    Used in Eq. 3 to pretrain the visual encoder; assumes the pseudo-gloss labels are temporally orderable against the video.
  • domain assumption Knowledge distillation from a visual-textual teacher to a visual-only student improves visual-only generation.
    VPD loss Eq. 5 relies on the standard assumption that matching teacher logits transfers knowledge; the paper provides only a gradient plausibility argument in Eq. 12.
  • domain assumption The target text is available as supervision for every training video and is a reliable ground truth for translation.
    Standard supervised SLT assumption; all loss terms use target text or text-derived pseudo-glosses.
  • domain assumption The Llama tokenizer and embedding space provide a suitable output interface for visual features after a linear projection.
    The architecture feeds visual features directly into Llama's input embedding space; no formal alignment guarantee is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs." pith.science (2026). https://pith.science/paper/65QZHW3Z

@misc{pith2026260809006,
  author       = {Pith},
  title        = {Pith review of: SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/65QZHW3Z}},
  note         = {Machine review of arXiv:2608.09006}
}
read the original abstract

Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks. However, fine-tuning LLMs for Gloss-Free Sign Language Translation (GFSLT) remains a challenge. In this paper, we investigate how to effectively adapt LLMs to the GFSLT task. We show that there are two key issues that need to be solved: (1) the inherent distributional gap between visual feature inputs and text feature inputs makes it difficult for LLMs to interpret visual inputs; and (2) existing approaches typically concatenate visual and textual features in an autoregressive framework, which leads to the model overemphasizing textual inputs and deprioritizing visual cues, as LLMs are pretrained predominantly on text-centric data. To address the first challenge, we propose a simple yet effective method named Filtered Pseudo-Gloss CTC Pretraining, which leverages filtered pseudo-gloss sequences generated from text sequences to supervise the training of the visual backbone. To tackle the second issue, we introduce a Visual-Prioritized Distillation training strategy. Specifically, we define a visual-only prediction path in which text inputs are masked, and the model is required to generate the target sequence relying solely on visual inputs. To guide this path, the outputs from the standard visual-textual prediction are then distilled into the visual-only prediction path, encouraging the model to prioritize visual features. Comprehensive experiments and qualitative analyses demonstrate the effectiveness of the proposed model. The proposed SignLlama achieves very competitive performance on multiple datasets for GFSLT tasks, without using any extra modalities or external sign language datasets for pretraining.

Figures

Figures reproduced from arXiv: 2608.09006 by the authors.

Figure 1
Figure 1. The proposed SignLlama with Visual-Prioritized Distillation [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Effect of different 𝜆 in VPD and different drop rates in FPG-CTC the Adam optimizer with a weight decay of 0.0001 for 50 epochs on three GeForce RTX 3090 GPUs. The initial learning rate is set to 1e-4 for the recognition model and 1e-5 for the translation model, and is decayed by a factor of 0.5 at epochs 10, 20 and 30. The batch size is set to 3. For FPG-CTC pretraining, the random dropout rate is set to 0.2. The l… view at source ↗
Figure 3
Figure 3. Visualization of feature distributions using t [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Visualization of the attention weights of Llama. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: SLT Qualitative results on Phoenix-2014T, CSL-Daily, How2Sign and OpenASL. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Visualization of attention maps in the last layer of Llama [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 6
Figure 6. Figure 6: Visualization of attention maps in the last layer of Llama with VPD, showing that the attention is mainly focused on [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Visualization of attention weight in the last layer of Llama without VPD, showing that the attention is mainly focused [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 26 canonical work pages

  1. [1]

    Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Deyao Zhu, Jian Ding, and Mohamed Elhoseiny. 2024. MiniGPT4-Video: Advancing Mul- timodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens. arXiv preprint arXiv:2404.03413(2024)

    arXiv 2024

  2. [2]

    Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. 2018. Neural sign language translation. InCVPR. 7784–7793

    work page 2018

  3. [3]

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechu Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023. MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning.arXiv preprint arXiv:2310.09478(2023)

    arXiv 2023

  4. [4]

    Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, and Brian Mak. 2022. Two-stream network for sign language recognition and translation.Advances in Neural Information Processing Systems35 (2022), 17043–17056

    2022

  5. [5]

    Zhigang Chen, Benjia Zhou, Yiqing Huang, Jun Wan, Yibo Hu, Hailin Shi, Yanyan Liang, Zhen Lei, and Du Zhang. 2025. C 2 RL: Content and Context Represen- tation Learning for Gloss-free Sign Language Translation and Retrieval.IEEE Transactions on Circuits and Systems for Video Technology(2025)

    work page 2025

  6. [6]

    Zhigang Chen, Benjia Zhou, Jun Li, Jun Wan, Zhen Lei, Ning Jiang, Quan Lu, and Guoqing Zhao. 2024. Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translation. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 7071–7081

    work page 2024

  7. [7]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.ICLR(2021)

    work page 2021

  8. [8]

    Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro-i Nieto. 2021. How2Sign: a large-scale multimodal dataset for continuous American sign language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2735–2744

    work page 2021

Show all 58 references
  1. [9]

    Shiwei Gan, Xiao Liu, Yafeng Yin, Nan Liu, Kuizhuang Liu, Desibieer Tuerdaken, Zhiwei Jiang, Lei Xie, Sanglu Lu, and Hongkai Wen. 2026. Learning Effective Sign Features without Text for Gloss-free Sign Language Translation. (2026)

    2026

  2. [10]

    Shiwei Gan, Lichen Wang, Xiao Liu, Yafeng Yin, Kuizhuang Liu, Sanglu Lu, and Lei Xie. 2026. Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding.arXiv preprint arXiv:2607.27826 (2026)

    2026 arXiv

  3. [11]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Hongkai Wen, Lei Xie, and Sanglu Lu. 2024. SignGraph: A Sign Sequence is Worth Graphs of Nodes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13470–13479

    2024

  4. [12]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Kang Xia, Lei Xie, and Sanglu Lu. 2023. Contrastive learning for sign language recognition and translation. InProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-

    2023

  5. [13]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Lei Xie, and Sanglu Lu. 2021. Skeleton- Aware Neural Sign Language Translation. InMM. 4353–4361

    2021

  6. [14]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Lei Xie, and Sanglu Lu. 2023. Towards Real- Time Sign Language Recognition and Translation on Edge Devices. InProceedings of the 31st ACM International Conference on Multimedia. 4502–4512

    2023

  7. [15]

    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Lei Xie, Sanglu Lu, and Hongkai Wen

  8. [16]

    Peter Gaskell, Jana Pavlasek, Tom Gao, Abhishek Narula, Stanley Lewis, and Odest Chadwicke Jenkins. 2024. MBot: A modular ecosystem for scalable robotics education. In2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 18294–18300

    2024

  9. [17]

    Jia Gong, Lin Geng Foo, Yixuan He, Hossein Rahmani, and Jun Liu. 2024. Llms are good sign language translators. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18362–18372

    2024

  10. [18]

    Michael Guevarra, Indronil Bhattacharjee, Srijita Das, Christabel Wayllace, Car- rie Demmans Epp, Matthew E Taylor, and Alan Tay. 2025. An LLM-Guided Tutoring System for Social Skills Training. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 29643–29645

    2025

  11. [19]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.ICLR1, 2 (2022), 3

    2022

  12. [20]

    Peiqi Jiao, Yuecong Min, and Xilin Chen. 2024. Visual alignment pre-training for sign language translation. InEuropean Conference on Computer Vision. Springer, 349–367

    2024

  13. [21]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning. PMLR, 19730–19742

    2023

  14. [22]

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355(2023)

    2023 arXiv

  15. [23]

    Han Liang, Chengyu Huang, Yuecheng Xu, Cheng Tang, Weicai Ye, Juze Zhang, Xin Chen, Jingyi Yu, and Lan Xu. 2024. LLaVA-SLT: Visual Language Tuning for Sign Language Translation.arXiv preprint arXiv:2412.16524(2024)

    2024 arXiv

  16. [24]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. InText summarization branches out. 74–81

    2004

  17. [25]

    Kezhou Lin, Xiaohan Wang, Linchao Zhu, Ke Sun, Yi Yang, et al. 2023. Gloss- Free End-to-End Sign Language Translation. InThe 61st Annual Meeting Of The Association For Computational Linguistics

    2023

  18. [26]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning.Advances in neural information processing systems36 (2023), 34892–34916

    2023

  19. [27]

    Xiao Liu, Shiwei Gan, Yafeng Yin, Bowen Guo, Zhiwei Jiang, Shunmei Meng, Lei Xie, and Sanglu Lu. 2026. SignPR: A Progressive Vector-Quantized Diffu- sion Framework for Sign Language Production. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2026

  20. [28]

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvinine- jad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation.Transactions of the Association for Computational Linguistics8 (2020), 726–742

    2020

  21. [30]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan

  22. [31]

    Yuecong Min, Aiming Hao, Xiujuan Chai, and Xilin Chen. 2021. Visual alignment constraint for continuous sign language recognition. InICCV. 11542–11551

    2021

  23. [32]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. InACL. 311–318

    2002

  24. [33]

    https://arxiv.org/abs/2406.09418

    VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding.arxiv(2024). https://arxiv.org/abs/2406.09418

    2024 arXiv

  25. [34]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners.OpenAI blog 1, 8 (2019), 9

    2019

  26. [35]

    Phillip Rust, Bowen Shi, Skyler Wang, Necati Cihan Camgöz, and Jean Maillard

  27. [36]

    Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat-Seng Chua, Yueting Zhuang, and Siliang Tang. 2024. Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning. arXiv:2402.11435 [cs.CV]

    2024 arXiv

  28. [37]

    Bowen Shi, Diane Brentari, Gregory Shakhnarovich, and Karen Livescu. 2022. Open-Domain Sign Language Translation Learned from Online Video. InProceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing. 6365–6379

    2022

  29. [38]

    Ozge Mercanoglu Sincan, Necati Cihan Camgoz, and Richard Bowden. 2023. Is context all you need? scaling neural sign language translation to large domains of discourse. InProceedings of the IEEE/CVF international conference on computer vision. 1955–1965

    2023

  30. [39]

    Towards privacy-aware sign language translation at scale.arXiv preprint arXiv:2402.09611(2024)

    2024 arXiv

  31. [40]

    Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020. BLEURT: Learning Robust Metrics for Text Generation. InProceedings of ACL

    2020

  32. [41]

    Dave Uthus, Garrett Tanzer, and Manfred Georg. 2023. Youtube-asl: A large-scale, open-domain american sign language-english parallel corpus.Advances in Neural Information Processing Systems36 (2023), 29029–29047

    2023

  33. [42]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research9, 11 (2008)

    2008

  34. [43]

    Laia Tarrés, Gerard I Gállego, Amanda Duarte, Jordi Torres, and Xavier Giró-i Nieto. 2023. Sign language translation from instructional videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5624–5634

    2023

  35. [44]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    2023 arXiv

  36. [45]

    Xin Xiao, Bohong Wu, Jiacong Wang, Chunyuan Li, Haoyuan Guo, et al. 2024. See- ing the image: Prioritizing visual correlation by contrastive alignment.Advances in Neural Information Processing Systems37 (2024), 30925–30950

    2024

  37. [46]

    Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont- Tuset, Ivan Laptev, Josef Sivic, and Cordelia Schmid. 2023. Vid2seq: Large-scale pretraining of a visual language model for dense video captioning. InProceedings of the IEEE/CVF Conference on Computer V...

    2023

  38. [47]

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Song XiXuan, et al. 2024. Cogvlm: Visual expert for pretrained language models.Advances in Neural Information Processing Systems MM ’26, November 10–14, 2026, Rio de Janeiro, B...

    2024

  39. [48]

    Ryan Wong, Necati Cihan Camgoz, and Richard Bowden. 2024. SIGN2GPT: leveraging large language models for gloss-free sign language translation. In ICLR 2024: The Twelfth International Conference on Learning Representations

    2024

  40. [49]

    Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, and Shuicheng Yan. 2022. Metaformer is actually what you need for vision. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10819–10829

    2022

  41. [50]

    Biao Zhang, Garrett Tanzer, and Orhan Firat. 2024. Scaling sign language transla- tion.Advances in neural information processing systems37 (2024), 114018–114047

    2024

  42. [51]

    Jinhui Ye, Xing Wang, Wenxiang Jiao, Junwei Liang, and Hui Xiong. 2024. Improv- ing Gloss-free Sign Language Translation by Reducing Representation Density. InNeurIPS

    2024

  43. [52]

    Aoxiong Yin, Tianyun Zhong, Li Tang, Weike Jin, Tao Jin, and Zhou Zhao. 2023. Gloss attention for gloss-free sign language translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2551–2562

    2023

  44. [53]

    Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. 2021. Improv- ing Sign Language Translation with Monolingual Data by Sign Back-Translation. InCVPR. 1316–1325

    2021

  45. [54]

    Qidan Zhu, Jing Li, Fei Yuan, and Quan Gan. 2024. Multiscale temporal network for continuous sign language recognition.Journal of Electronic Imaging33, 2 (2024), 023059–023059

    2024

  46. [55]

    Jihai Zhang, Wengang Zhou, and Houqiang Li. 2014. A threshold-based hmm-dtw approach for continuous sign language recognition. InICIMCS. 237–240

    2014

  47. [56]

    Benjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. 2023. Gloss-free sign language translation: Improving from visual-language pretraining. InProceedings of the IEEE/CVF International Conference on Computer Vision. 20871–20881

    2023

  48. [59]

    Ronglai Zuo and Brian Mak. 2022. C2SLR: Consistency-Enhanced Continuous Sign Language Recognition. InCVPR. 5131–5140. SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil A Datas...

    2022

  49. [2024]

    InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

    Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

    2024

  50. [2025]

    MixSignGraph: A Sign Sequence is Worth Mixed Graphs of Nodes

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.