Pith. sign in

REVIEW 2 major objections 6 minor 49 references

VTaMo: Video-Text Alignment Model for Sign Language Translation

T0 review · 2 major / 6 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Explicit multi-level vision-text alignment turns gloss-free sign language translation into a reordering-and-grounding problem and reaches state-of-the-art on four benchmarks.

desk verdict Solid SOTA gloss-free SLT via explicit multi-granularity alignment; the pieces are known, the joint adaptation to non-monotonic partial sign correspondence is real and well-supported. read the letter →

arxiv 2607.09126 v1 pith:7RJIJKGF submitted 2026-07-10 cs.CV cs.CL

Junyi Hu , Zhewen He , Haomian Huang , Aoxiang Yang , Yi Fang This is my paper · ORCID
classification cs.CVcs.CL
keywords signlanguagetranslationcross-modalalignmentoptimaltransportgloss-freeSLTcontrastivelearningEarthMover'sDistancevideo-text
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Gloss-free sign language translation must map continuous signing video to spoken sentences without intermediate gloss labels, while sign order often diverges from spoken word order. Most systems leave that matching to the decoder’s attention. This paper argues that making the match explicit—at frame-to-token, sentence, and token levels—before decoding is enough to reach strong translation quality without visual-encoder fine-tuning or large-scale sign pre-training. VTaMo does this with entropy-regularized optimal transport plus a null token for local matches, an orthogonal map and Earth Mover’s Distance for global geometry, and position-aligned contrastive learning after reordering. Across German, Chinese, and English benchmarks the gains hold, and ablations show each alignment piece contributes.

What carries the argument

Multi-granularity vision-text alignment: local Sinkhorn optimal transport with a learnable null token that absorbs transitional frames, a learnable orthogonal transform that calibrates sentence embedding geometry via Earth Mover’s Distance, and position-aligned contrastive learning on window-reordered visual features. The resulting transport plan reorders visual features into target-token order for the decoder during training.

What would settle it

Retrain the same architecture while aligning and decoding full sentences (no pseudo-gloss filter) on Phoenix-2014T and How2Sign; if BLEU-4 collapses and transport plans lose block-diagonal structure, the intermediate content-word premise fails.

Watch

Extended reading notes

Core claim

VTaMo claims that explicit multi-granularity cross-modal alignment—local entropy-regularized optimal transport with a learnable null token, global orthogonal calibration via Earth Mover’s Distance, and position-aligned contrastive learning—consistently improves gloss-free sign language translation and yields state-of-the-art results on Phoenix-2014T, CSL-Daily, How2Sign, and OpenASL without visual fine-tuning or extra sign pre-training.

Load-bearing premise

The method depends on a fixed part-of-speech filter that strips spoken sentences to content-word “pseudo-gloss” as a fair intermediate target for alignment and decoding across languages, so a pure-text model can later restore function words and spoken order.

Editorial extensions

If this is right

  • Gloss-free SLT can approach gloss-based accuracy when alignment is made explicit rather than left to decoder attention.
  • Gains widen on large-vocabulary, open-domain sets, so explicit alignment scales with linguistic complexity.
  • A frozen CLIP visual backbone plus a LoRA-adapted language model is enough when the three alignment losses are present.
  • Learned transport plans become interpretable block-diagonal maps linking contiguous signing segments to content tokens.
  • A text-only recovery model can restore function words and spoken order after the decoder emits video-order pseudo-gloss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The null-token OT pattern may transfer to other partial, non-monotonic video-text tasks such as instructional step grounding.
  • If the POS filter is language-sensitive, multilingual systems may need language-aware content filters rather than one rule set.
  • Real-time or streaming SLT would need an approximate online form of Sinkhorn reordering, since the current plan is sentence-level and offline.
  • Joint visual fine-tuning or stronger video backbones would likely amplify rather than replace the alignment gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. VTaMo proposes gloss-free sign language translation with explicit multi-granularity vision–text alignment rather than relying solely on decoder attention. The method (i) solves entropy-regularized optimal transport between temporal visual features and content-word pseudo-gloss embeddings, with a learnable null token for transitional frames; (ii) calibrates sentence-level embedding geometry via a learnable orthogonal map and Earth Mover’s Distance over a memory queue; and (iii) applies position-aligned contrastive learning on window-reordered visual tokens. Training uses OT-guided reordering so the decoder sees text-ordered visual features; at inference the decoder emits pseudo-gloss in signing order and a text-only recovery model restores fluent spoken order and function words. On Phoenix-2014T, CSL-Daily, How2Sign, and OpenASL the system reports state-of-the-art gloss-free BLEU/ROUGE/BLEURT without visual-encoder fine-tuning or large-scale sign pretraining, with component ablations, backbone controls, alignment-quality metrics, and a ControlNet-based noise study supporting the design.

Significance. If the results hold under the stated protocol, the paper makes a clear and transferable contribution: explicit local OT (with null assignment), global orthogonal EMD calibration, and position-aligned contrastive learning are complementary and improve gloss-free SLT across three languages and four public benchmarks, including large-vocabulary English sets where prior gloss-free systems struggle. Gains without visual fine-tuning or sign-specific pretraining, plus released code, strengthen the claim that structured cross-modal alignment is a practical alternative to scaling data or pose streams alone. Ablations (Tables 4–7), transport-plan diagnostics (Table 6), and appearance/background robustness (Table 8) go beyond typical single-number SOTA reports and make the work useful as a reference design for non-monotonic, partial video–text correspondence.

major comments (2)
  1. Sec. 3.1 and the inference paragraph: final metrics are computed after a text-only recovery model that reorders content words and re-inserts function words. The central claim is that multi-granularity visual–text alignment improves SLT, yet the contribution of recovery versus correct content-word prediction is not isolated. Please report at least one of: (i) BLEU/ROUGE on decoder pseudo-gloss against reference pseudo-gloss (before recovery); (ii) content-word F1 or order-insensitive bag-of-content-word scores; or (iii) an oracle recovery upper bound versus the trained recovery model. Without this, it is hard to attribute how much of the SOTA margin is alignment versus text-side restoration, especially on How2Sign/OpenASL where word order and function words matter for BLEU.
  2. Sec. 3.5 and Eq. (13): training reorders visual windows using the OT plan against ground-truth pseudo-gloss embeddings, while inference never reorders. Table 4 shows large drops when removing L_local or L_contra, but there is no controlled experiment that keeps all losses and disables reordering only (or replaces OT reordering with random/monotonic windows). A short ablation on Phoenix-2014T would confirm that the reported gains require the OT-guided reorder operator rather than the alignment losses alone acting as regularizers on an unordered stream.
minor comments (6)
  1. Tables 1–3 report single-run scores with no seed variance or significance tests. For the largest claimed margins (e.g., +4.54 BLEU-4 vs SpaMo on Phoenix-2014T, +8.36 on How2Sign), 2–3 seeds or bootstrap intervals would strengthen the SOTA claim.
  2. Sec. 4.2 lists many free hyperparameters (ε annealing, ρ_target, β weights, queue size, global ramp). A brief sensitivity sweep on Phoenix-2014T for ε schedule and ρ_target would help reproducibility beyond the released code.
  3. Fig. 4 and Appendix Fig. S1: transport plans are informative; adding a short quantitative comparison of peak/entropy/change metrics on How2Sign or OpenASL (not only Phoenix-2014T development, Table 6) would show that alignment quality generalizes.
  4. Sec. 2.2 and Appendix Table S2: the generic video–text baselines (CLIP4Clip, VideoCLIP, X-CLIP) are useful negative controls; state clearly whether they received the same pseudo-gloss targets and LoRA decoder or only clip-level losses, so the comparison is not confounded by target representation.
  5. Limitations (Sec. 6) correctly notes offline evaluation and text-only recovery; a sentence on latency of Sinkhorn at inference (local OT is training-only for reordering, but clarify what runs at test time) would help systems readers.
  6. Minor presentation: ensure consistent naming (VTaMo vs VT aMo in Fig. 1 caption), and that all equation symbols in Sec. 3.3–3.6 (Â, π_w, Ω_u, λ_g(t)) are defined before first use in the main text as well as the appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical multi-loss SLT model evaluated on held-out BLEU/ROUGE; alignment objectives do not define the reported metrics by construction.

full rationale

VTaMo is a standard empirical vision–language paper. Its central claim—that local Sinkhorn OT with a null token, global orthogonal EMD calibration, and position-aligned contrastive learning improve gloss-free SLT—is supported by training under a composite loss (Eq. 16) and measuring BLEU-n / ROUGE-L / BLEURT on official held-out test splits of Phoenix-2014T, CSL-Daily, How2Sign, and OpenASL. Hyperparameters (ε annealing, λ weights, ρ_target, memory-queue size) are development choices; they do not algebraically force the reported test numbers. Component ablations (Table 4), backbone controls under fixed LMs (Table 5), alignment-quality diagnostics (Table 6), and noise-sensitivity checks (Table 8) are independent of the loss definitions. The spaCy pseudo-gloss filter and text-only recovery model are modeling choices applied uniformly offline; they do not make the SOTA comparison tautological. Related-work citations are to external methods (SpaMo, Uni-Sign, SHuBERT, etc.), not load-bearing self-citations of uniqueness theorems or fitted constants renamed as predictions. No self-definitional loop, fitted-input-as-prediction, or ansatz-via-self-citation is present. Score 0 is the correct honest finding.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central empirical claim rests on standard ML practice plus a handful of hand-chosen schedules and the pseudo-gloss modeling choice. No new physical entities; free parameters are ordinary training hyper-parameters whose values are reported.

free parameters (4)
  • ε annealing schedule (ε_high=0.12 → ε_mid=0.10 → ε_low=0.03)
    Controls sharpness of local OT assignments; chosen by schedule rather than derived.
  • null-ratio target ρ_target=0.2 and bias b_∅
    Hand-set target that keeps the learnable null token from dominating; directly affects alignment quality metrics.
  • loss weights (β_local=2.0, β_tv=0.1, β_null=0.1, λ_local=1.0, λ_g^max=0.1, β_orth=0.05, β_contra=1.0, τ_c=0.1)
    Scalar multipliers that balance the multi-objective loss; selected on development performance.
  • memory-queue capacity 256 and global activation ramp (4 000 steps)
    Engineering choices that determine the diversity and timing of the sentence-level EMD term.
assumptions (4)
  • domain assumption Cosine distance between projected visual features and frozen LM token embeddings is a meaningful transport cost for sign-level semantics.
    Stated in Sec. 3.3; underpins the entire local OT formulation.
  • ad hoc to paper A fixed spaCy POS filter yields a language-agnostic pseudo-gloss that is a sufficient intermediate target for both alignment and decoding.
    Sec. 3.1; the same rule is applied across German/Chinese/English without linguistic validation of coverage.
  • domain assumption An orthogonal linear map is sufficient to correct coordinate mismatch between pre-trained visual and textual sentence embeddings without distorting geometry.
    Sec. 3.4; motivated by cross-lingual embedding literature but not proved for sign features.
  • standard math Sinkhorn entropy-regularized OT with uniform marginals yields a usable soft alignment for non-monotonic, partial frame-to-token maps.
    Standard OT tool (Cuturi 2013) applied in Sec. 3.3.
invented entities (2)
  • learnable null token e_∅ with bias b_∅
    purpose: Absorbs transitional/co-articulation frames that have no lexical counterpart so the transport plan is not forced to be total.
    Introduced in Sec. 3.3; independent evidence is only the ablation and alignment-quality metrics inside this paper.
  • window-based reordering operator guided by the OT plan
    purpose: Presents the decoder with a visual sequence whose order matches the target pseudo-gloss during training.
    Sec. 3.5; engineering construct whose necessity is shown by the training-objective mismatch argument.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VTaMo: Video-Text Alignment Model for Sign Language Translation." pith.science (2026). https://pith.science/paper/7RJIJKGF

@misc{pith2026260709126,
  author       = {Pith},
  title        = {Pith review of: VTaMo: Video-Text Alignment Model for Sign Language Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7RJIJKGF}},
  note         = {Machine review of arXiv:2607.09126}
}
read the original abstract

Sign language translation (SLT) converts continuous sign videos into spoken language text. Gloss-free approaches leverage pre-trained visual encoders and language models but rely on implicit cross-modal alignment from translation supervision alone. We present VTaMo, a framework that introduces explicit multi-granularity alignment at three levels: (1) local alignment via entropy-regularized optimal transport with a learnable null token for fine-grained frame-to-token correspondences; (2) global alignment via a learnable orthogonal transformation that calibrates embedding space geometry through Earth Mover's Distance; and (3) position-aligned contrastive learning for discriminative token-level representations. Experiments on Phoenix-2014T, CSL-Daily, How2Sign, and OpenASL demonstrate consistent state-of-the-art performance, with ablations confirming the complementary contributions of each component. Code is available at https://github.com/junyi2005/vtamo.

Figures

Figures reproduced from arXiv: 2607.09126 by the authors.

Figure 1
Figure 1. Motivation and performance of VTaMo. (a) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. VTaMo pipeline. A sign-language video is encoded into visual tokens using a frozen CLIP-ViT backbone followed by a lightweight temporal encoder and fusion pro￾jection (A,B). Given the ground-truth text embeddings (D), VTaMo performs local alignment with an entropy-regularized OT solver (Sinkhorn) to obtain soft token– frame matches, and global alignment by mapping sign features into the text space with an orthogonal… view at source ↗
Figure 3
Figure 3. Representative augmented samples used in the robustness analysis on How2Sign [11]. (a) and (c) are background-replacement conditions generated via text￾prompt editing; (b) is a full body shape and identity transfer to a synthetic signer generated via text prompt. Hand and body kinematics are preserved via ControlNet pose conditioning. The synthetic signer in (b) does not represent any real individual. All modificati… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative analysis on How2Sign [11] test samples. (a) OT transport plans from VTaMo’s Sinkhorn solver: rows are visual frames and columns are pseudo-gloss tokens, with darker cells indicating higher transport mass; the block-diagonal struc￾ture reflects temporally co…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 3 linked inside Pith

  1. [1]

    In: CVPR (2018)

    Camgöz, N.C., Hadfield, S., Koller, O., Ney, H., Bowden, R.: Neural sign language translation. In: CVPR (2018)

    2018

  2. [2]

    In: CVPR (2020)

    Camgöz, N.C., Koller, O., Hadfield, S., Bowden, R.: Sign language transformers: Joint end-to-end sign language recognition and translation. In: CVPR (2020)

    2020

  3. [3]

    In: ICLR (2023)

    Chen, G., Yao, W., Song, X., Li, X., Rao, Y., Zhang, K.: PLOT: Prompt learning with optimal transport for vision-language models. In: ICLR (2023)

    2023

  4. [4]

    In: CVPR

    Chen, Y., Wei, F., Sun, X., Wu, Z., Lin, S.: A simple multi-modality transfer learning baseline for sign language translation. In: CVPR. pp. 5120–5130 (2022)

    2022

  5. [5]

    In: NeurIPS

    Chen, Y., Zuo, R., Wei, F., Wu, Y., Liu, S., Mak, B.: Two-stream network for sign language recognition and translation. In: NeurIPS. vol. 35, pp. 17043–17056 (2022)

    2022

  6. [6]

    IEEE Transactions on Circuits and Systems for Video Technology (2025)

    Chen, Z., Zhou, B., Huang, Y., Wan, J., Hu, Y., Shi, H., Liang, Y., Lei, Z., Zhang, D.: C2RL: Content and context representation learning for gloss-free sign language translation and retrieval. IEEE Transactions on Circuits and Systems for Video Technology (2025)

    2025

  7. [7]

    In: Proceedings of the 2024 Joint International Conference on Compu- tational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)

    Chen, Z., Zhou, B., Li, J., Wan, J., Lei, Z., Jiang, N., Lu, Q., Zhao, G.: Fac- torized learning assisted with large language model for gloss-free sign language translation. In: Proceedings of the 2024 Joint International Conference on Compu- tational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). pp. 7071–7081. ELRA and ICCL (2024)

    2024

  8. [8]

    In: NeurIPS (2024)

    Chowdhury, S.S., Chandra, S., Roy, K.: OPEL: Optimal transport guided proce- dure learning. In: NeurIPS (2024)

    2024

Show all 49 references
  1. [9]

    JMLR25(70), 1–53 (2024)

    Chung, H.W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al.: Scaling instruction-finetuned language models. JMLR25(70), 1–53 (2024)

    2024

  2. [10]

    In: NeurIPS (2013)

    Cuturi, M.: Sinkhorn distances: Lightspeed computation of optimal transport. In: NeurIPS (2013)

    2013

  3. [11]

    In: CVPR (2021)

    Duarte, A., Palaskar, S., Ventura, L., Ghadiyaram, D., DeHaan, K., Metze, F., Tor- res, J., Giro-i Nieto, X.: How2sign: A large-scale multimodal dataset for continuous american sign language. In: CVPR (2021)

    2021

  4. [12]

    In: ICASSP

    Fu,B., Ye, P.,Zhang,L.,Yu,P.,Hu, C.,Shi, X.,Chen, Y.:Atoken-level contrastive framework for sign language translation. In: ICASSP. pp. 1–5 (2023)

    2023

  5. [13]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Gong, J., Foo, L.G., He, Y., Rahmani, H., Liu, J.: Llms are good sign language translators. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18362–18372 (2024)

    2024

  6. [14]

    Gueuwou, S., Du, X., Shakhnarovich, G., Livescu, K., Liu, A.H.: SHuBERT: Self- supervised sign language representation learning via multi-stream cluster predic- tion. In: ACL. pp. 28792–28810 (2025)

    2025

  7. [15]

    In: ICLR (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. In: ICLR (2022)

    2022

  8. [16]

    In: NAACL (2025)

    Hwang, E.J., Cho, S., Lee, J., Park, J.C.: An efficient gloss-free sign language trans- lation using spatial configurations and motion dynamics with llms. In: NAACL (2025)

    2025

  9. [17]

    In: ECCV (2024)

    Jiao, P., Min, Y., Chen, X.: Visual alignment pre-training for sign language trans- lation. In: ECCV (2024)

    2024

  10. [18]

    In: ICASSP

    Jing, L., Song, X., Zu, X., Zheng, N., Zhao, Z., Nie, L.: Vk-g2t: Vision and context knowledge enhanced gloss2text. In: ICASSP. pp. 7860–7864 (2024)

    2024

  11. [19]

    In: ICLR (2018) VTaMo 17

    Lample, G., Conneau, A., Ranzato, M., Denoyer, L., Jégou, H.: Word translation without parallel data. In: ICLR (2018) VTaMo 17

    2018

  12. [20]

    In: NeurIPS

    Li,D.,Xu,C.,Yu,X.,Zhang,K.,Swift,B.,Suominen,H.,Li,H.:Tspnet:Hierarchi- cal feature learning via temporal semantic pyramid for sign language translation. In: NeurIPS. vol. 33, pp. 12034–12045 (2020)

    2020

  13. [21]

    In: ICLR (2025)

    Li, Z., Zhou, W., Zhao, W., Wu, K., Hu, H., Li, H.: Uni-sign: Toward unified sign language understanding at scale. In: ICLR (2025)

    2025

  14. [22]

    In: Text Sum- marization Branches Out (2004)

    Lin, C.Y.: Rouge: A package for automatic evaluation of summaries. In: Text Sum- marization Branches Out (2004)

    2004

  15. [23]

    Lin, K., Wang, X., Zhu, L., Sun, K., Zhang, B., Yang, Y.: Gloss-free end-to-end sign language translation. In: ACL. pp. 12904–12916 (2023)

    2023

  16. [24]

    In: ICLR (2019)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019)

    2019

  17. [25]

    Neuro- computing508, 293–304 (2022)

    Luo, H., Ji, L., Zhong, M., Chen, Y., Lei, W., Duan, N., Li, T.: CLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioning. Neuro- computing508, 293–304 (2022)

    2022

  18. [26]

    In: ACM MM

    Ma, Y., Xu, G., Sun, X., Yan, M., Zhang, J., Ji, R.: X-CLIP: End-to-end multi- grained contrastive learning for video-text retrieval. In: ACM MM. pp. 638–647 (2022)

    2022

  19. [27]

    In: ICCV

    Mavroudi, E., Afouras, T., Torresani, L.: Learning to ground instructional articles in videos through narrations. In: ICCV. pp. 15201–15213 (2023)

    2023

  20. [28]

    Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M.S., Shen, S., Yong, Z.X., Schoelkopf, H., Tang, X., Radev, D., Aji, A.F., Almubarak, K., Albanie, S., Alyafeai, Z., Webson, A., Raff, E., Raffel, C.: Crosslin- gual generalization through ...

    2023

  21. [29]

    arXiv preprint arXiv:1807.03748 (2018)

    van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

    2018 arXiv

  22. [30]

    In: ACL (2002)

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: A method for automatic evaluation of machine translation. In: ACL (2002)

    2002

  23. [31]

    In: ICML (2021)

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021)

    2021

  24. [32]

    In: CVPR (2022)

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: CVPR (2022)

    2022

  25. [33]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2024)

    Rust, P., Shi, B., Wang, S., Camgöz, N.C., Maillard, J.: Towards privacy-aware sign language translation at scale. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2024)

    2024

  26. [34]

    In: ACL (2020)

    Sellam, T., Das, D., Parikh, A.: BLEURT: Learning robust metrics for text gener- ation. In: ACL (2020)

    2020

  27. [35]

    Shi, B., Wu, Z., Mao, M., Wang, X., Darrell, T.: When do we not need larger vision models? In: ECCV (2024)

    2024

  28. [36]

    In: EMNLP (2022)

    Shi, B., Brentari, D., Shakhnarovich, G., Livescu, K.: Open-domain sign language translation learned from online video. In: EMNLP (2022)

    2022

  29. [37]

    arXiv preprint arXiv:2008.00401 (2020)

    Tang, Y., Tran, C., Li, X., Chen, P.J., Goyal, N., Chaudhary, V., Gu, J., Fan, A.: Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint arXiv:2008.00401 (2020)

    2008 arXiv

  30. [38]

    arXiv preprint arXiv:2307.09288 (2023)

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bash- lykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  31. [39]

    In: NeurIPS

    Uthus, D., Tanzer, G., Georg, M.: Youtube-asl: A large-scale, open-domain amer- ican sign language-english parallel corpus. In: NeurIPS. vol. 36 (2023)

    2023

  32. [40]

    In: ICLR (2024) 18 J

    Wong, R., Camgoz, N.C., Bowden, R.: Sign2gpt: Leveraging large language models for gloss-free sign language translation. In: ICLR (2024) 18 J. Hu et al

    2024

  33. [41]

    In: EMNLP

    Xu, H., Ghosh, G., Huang, P.Y., Okhonko, D., Aghajanyan, A., Metze, F., Zettle- moyer, L., Feichtenhofer, C.: VideoCLIP: Contrastive pre-training for zero-shot video-text understanding. In: EMNLP. pp. 6787–6800 (2021)

    2021

  34. [42]

    In: ACM MM

    Yin, A., Zhao, Z., Liu, J., Jin, W., Zhang, M., Zeng, X., He, X.: Simul-slt: End-to- end simultaneous sign language translation. In: ACM MM. pp. 4118–4127 (2021)

    2021

  35. [43]

    In: CVPR

    Yin, A., Zhong, T., Tang, L., Jin, W., Jin, T., Zhao, Z.: Gloss attention for gloss- free sign language translation. In: CVPR. pp. 2551–2562 (2023)

    2023

  36. [44]

    In: ICLR (2023)

    Zhang, B., Müller, M., Sennrich, R.: Sltunet: A simple unified model for sign lan- guage translation. In: ICLR (2023)

    2023

  37. [45]

    In: ICCV (2023)

    Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: ICCV (2023)

    2023

  38. [46]

    IEEE TMM 24, 2662–2672 (2021)

    Zhao, J., Qi, W., Zhou, W., Duan, N., Zhou, M., Li, H.: Conditional sentence generation and cross-modal reranking for sign language translation. IEEE TMM 24, 2662–2672 (2021)

    2021

  39. [47]

    In: ICCV

    Zhou, B., Chen, Z., Clapés, A., Wan, J., Liang, Y., Escalera, S., Lei, Z., Zhang, D.: Gloss-free sign language translation: Improving from visual-language pretraining. In: ICCV. pp. 20871–20881 (2023)

    2023

  40. [48]

    In: CVPR

    Zhou, H., Zhou, W., Qi, W., Pu, J., Li, H.: Improving sign language translation with monolingual data by sign back-translation. In: CVPR. pp. 1316–1325 (2021)

    2021

  41. [49]

    IEEE TMM24, 768–779 (2022) VTaMo 19 Appendix This appendix provides expanded details and additional results complementing the main paper

    Zhou, H., Zhou, W., Zhou, Y., Li, H.: Spatial-temporal multi-cue network for sign language recognition and translation. IEEE TMM24, 768–779 (2022) VTaMo 19 Appendix This appendix provides expanded details and additional results complementing the main paper. Section S.1 present...

    2022

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.