Pith. sign in

REVIEW 3 major objections 5 minor 65 references

A six-field TSV format plus configuration file lets sign-language translation run on Hugging Face with no custom training code.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

MultimodalHugs provides a standardized TSV-based dataset format, modular processors, and Hugging Face integration to enable reproducible sign language and multimodal translation experiments.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A useful framework idea undermined by overclaimed evidence: the only executed reproduction shows an unexplained BLEU gap, and the SOTA 'reproduction' was never actually run. the 3 major comments →

arxiv 2509.09729 v1 pith:3PXS5Y5W submitted 2025-09-10 cs.CL cs.AIcs.MM

MultimodalHugs: Enabling Sign Language Processing in Hugging Face

Gerard Sant , Zifan Jiang , Carlos Escolano , Amit Moryossef , Mathias M\"uller , Rico Sennrich , Sarah Ebling This is my paper
classification cs.CL cs.AIcs.MM
keywords sign language processingmultimodal translationHugging Facereproducibilitypose estimationpixel-based machine translationTSV dataset formatmultimodal framework
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that sign language processing lags behind text NLP because of tooling, not algorithms: experiments rest on ad-hoc scripts that are hard to reproduce. MultimodalHugs adds a standardization layer to the Hugging Face ecosystem, describing every example as one TSV row and every training run as one YAML file, so switching inputs—pose sequences, precomputed video features, raw video, or pixel-rendered words—means changing a processor and a config, not writing new code. The authors back this with two demonstrations: they reproduce a published pose-to-text sign language system and extend it to I3D features and end-to-end video, and they apply the same pipeline to Hebrew-to-English translation from images of words, matching a tokenizer-based baseline with 45% fewer tokens. If the schema is as expressive as claimed, SLP gains a common substrate for reproducible, comparable experiments.

Core claim

The paper's central claim is that a fixed TSV schema—signal, signal_start, signal_end, encoder_prompt, decoder_prompt, output—plus a YAML configuration is enough to express a wide class of sign language and multimodal translation experiments, so a single untampered training loop can handle pose data, video features, raw frames, and rendered text. The processor abstraction decouples modality-specific preprocessing from the model, making a modality or backbone swap a configuration change rather than a code-writing task. Evidence includes a faithful reproduction of the Uthus et al. (2023) pose-to-text system, improved results when pose is replaced by I3D features, and a pixel-based Hebrew-to-En

What carries the argument

The load-bearing object is the six-field TSV row schema combined with the modular processor abstraction. The TSV forces every dataset into one shape so a single Hugging Face Trainer loop can run any task; the processor converts whatever the signal field points to (e.g., a .pose file, .mp4, or image sequence) into the model's input tensors. A YAML config instantiates the training actors—dataset, processor, model—and selects the model's components (feature extractor, multimodal mapper, backbone), so setup, training, and generation are three command-line calls.

Load-bearing premise

The six-field TSV schema plus the available processors is assumed expressive enough to capture any realistic sign language experiment; if a task needs multiple simultaneous signals or an alignment the schema cannot represent, the 'no additional code' promise fails.

What would settle it

Run the Appendix C reproduction of Zhang et al. (2024) to completion—SLT, alignment, machine translation, and augmented SLT—from only the TSV and YAML files and compare BLEU/chrF with the published numbers. The paper states this run was never completed, so any divergence, required custom code, or schema limit (e.g., a sample needing more than one embedded signal) would disprove the central claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A published SLP system can be re-run by preparing TSV files and a YAML file, with no custom training scripts.
  • Modality comparisons (pose vs. I3D features vs. raw video) become cheap and fair, since only the processor and dataset pointers change.
  • The same infrastructure extends beyond sign languages to tasks where input can be rendered as frames, such as pixel-based translation for morphologically rich scripts.
  • By inheriting Hugging Face's Trainer, the framework brings mixed-precision, distributed training, and standard metrics to SLP experiments.
  • Full automation of setup/train/generate makes the entire experiment—data, preprocessing, and evaluation—auditable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The schema allows only one primary signal per example, so a task needing simultaneous or interleaved modalities (e.g., pose plus video in one input) still requires custom code today; the paper's own future-work section points to this limit.
  • The Hebrew pixel experiment suggests visual tokenization could be a general workaround for morphology-heavy languages, providing an efficiency lever (45% fewer tokens) beyond just sign languages.
  • If the TSV format becomes conventional, dataset-level comparison improves and the 'code not published' failure mode the paper surveys becomes easy to audit, since a TSV file itself is the interface.
  • The universality of 'no additional code' is bounded by the available processor library: each genuinely new modality requires a new processor, so the framework's reach grows with that library.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MultimodalHugs, a framework built on Hugging Face that aims to make multimodal sign language processing (SLP) reproducible and easy by standardizing datasets as TSV files, using modular modality-specific processors, and controlling training/evaluation through YAML configuration and three CLI commands (setup, train, generate). The authors argue that current SLP code is ad hoc and that Hugging Face lacks support for non-text inputs. They demonstrate the framework on two tasks: (1) sign-to-text translation on How2Sign using MediaPipe poses, I3D features, and raw video, with mT5 and M2M-100 backbones; and (2) Hebrew-to-English translation where Hebrew words are rendered as pixel images and embedded with CLIP. Appendix C claims to reproduce the multitask SOTA system of Zhang et al. (2024) without custom code, but explicitly states the experiment was not run to completion. The central claim of a general no-custom-code framework is therefore backed by two relatively simple executed demonstrations and an unexecuted complex reproduction.

Significance. If the framework performs as claimed, it would address a real and widely felt need in SLP: reducing ad-hoc code, enabling fairer comparisons, and integrating with the Hugging Face ecosystem. The paper's strengths include a clean conceptual abstraction (TSV + processors + YAML), open-source release, multiple executed experiments spanning different modalities, and transparent reporting in Appendix A. However, two issues affect the central claim. First, the only executed comparison with prior work (§4.1, Table 1) shows an unexplained BLEU gap: the supposed faithful reproduction of Uthus et al. gives 1.43 BLEU versus the original 0.86. Second, the strongest evidence for the 'no additional code' claim—the SOTA reproduction in Appendix C—was not actually run, so the four-task multitask setup and its mixing/weighting/evaluation remain unverified. These gaps are fixable but currently undermine the paper's most prominent promise.

major comments (3)
  1. [§4.1, Table 1] The reproduction claim for Uthus et al. (2023) is not supported by the reported numbers. Row 0 reports the original result of BLEU 0.86; row 1 reports the MultimodalHugs reproduction at BLEU 1.43, a roughly 66% relative improvement. The text states this 'replicates the original system' without any explanation for the gap. If the same data splits, preprocessing, evaluation script, and checkpoint selection were used, the discrepancy itself needs explanation. If the setup differs in any way (e.g., use of How2Sign versus other data, tokenization, case handling, beam size, seed, or epoch count), those differences must be stated. As written, this undermines the 'faithful reproduction' claim and affects the credibility of the other rows.
  2. [Appendix C.3, §3.1] The claim that MultimodalHugs can 'simply reproduce' the state-of-the-art multitask SLT system of Zhang et al. (2024) without custom code is not experimentally verified. Appendix C.3 explicitly states: 'Although we did not run this experiment to completion due to computational constraints...'. The appendix shows how the four tasks (SLT, alignment, MT, augmented SLT) can be mapped to the TSV fields, but it does not demonstrate that the seq2seq task actually handles the four distinct objectives, their mixing, weighting, or evaluation. Since this is the only complex multitask demonstration in the paper, the 'without requiring any custom scripts' claim is unsupported. I recommend either running at least a reduced-scale or component-wise version of the multitask experiment, or explicitly re-scoping the claim to 'designed to support' rather than 'reproduces'.
  3. [§3.1, §4] The paper's core expressiveness claim—that a fixed six-field TSV schema plus the modular processor abstraction can capture 'essentially any' multimodal SLP experiment without task-specific code—is demonstrated only on two tasks: pose/video-to-text SLT and Hebrew pixel-based MT. Both tasks share the same input modality (a single signal sequence) and text output. The framework's ability to handle multiple simultaneous signals (e.g., pose + video), non-text outputs, or tasks requiring alignment losses is not shown. This is closely tied to the unexecuted Appendix C: the most complex scenario that would exercise these capabilities is precisely the one that was not run. Please either provide at least one additional executed experiment with a structurally different input/output configuration, or temper the generality claims in §1 and §3.1.
minor comments (5)
  1. [Abstract, §3.1] Grammar: 'Appendix C shows how MultimodalHugs simply reproduction of a complex...' should be 'simply reproduces'. Similar phrasing in the abstract could be tightened.
  2. [§3.4] There is a garbled line of characters immediately after the multimodalhugs-generate code block ('.אר×פשוטØÚ'). This appears to be a rendering/encoding artifact and should be removed.
  3. [References] Author name formatting issues: 'Leandro V on Werra' should be 'Leandro Von Werra'; the entry for 'Yin Kayo and Read Jesse' should be alphabetized under 'Yin' and formatted as 'Kayo Yin and Jesse Read'.
  4. [Appendix A] The survey is small (n=16) and self-selected; the use of phrases such as 'strong interest' and 'would consider using it as their main framework' should be framed as informal usability feedback rather than a generalizable empirical result. The summary already calls it 'informal', which is good, but the main text should mirror this caveat.
  5. [Table 2] The metric 'XCOMET-XXL' is used without a citation or definition in the text. If it is a version of COMET, please cite the corresponding reference and specify which checkpoint was used.

Circularity Check

0 steps flagged

No circularity: the framework's claims are empirical/engineering assertions, and the unexecuted Appendix C reproduction and BLEU gap are verification gaps, not reductions to inputs.

full rationale

The paper is a software-framework contribution, not a derivation whose conclusions are fed back as assumptions. The two demonstrations are compared against external baselines (Uthus et al. 2023 for SLT; M2M-100 for Hebrew MT) using standard metrics, and the results do not reduce to the framework's own design choices. The SLT 'reproduction' is a training run on the same How2Sign data with the same linear-mT5 architecture; although Table 1 row 1 (BLEU 1.43) does not match row 0 (0.86), that is an unexplained discrepancy or reproducibility gap, not a circular reduction—the evaluation is independent of the framework's own predictions. Appendix C, which claims that the Zhang et al. SOTA system can be replicated with TSV+YAML, explicitly states: 'Although we did not run this experiment to completion due to computational constraints, this setup faithfully replicates the architecture and multitask training scheme described by Zhang et al. (2024), relying solely on standardized TSV metadata and a YAML configuration file.' That is an unexecuted verification claim, not a definitional identity. The survey in Appendix A is self-collected and supports the motivational premise, but it is not used as a formal premise in any claimed result, so it does not make the derivation circular. Self-citations (e.g., Müller et al. 2023a,b; Moryossef et al. 2023) point to external tools or prior findings and are corroborated by the paper's own experiments. No load-bearing claim reduces by construction to its own input.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

No free parameters are fitted to data; all results use standard pretrained models and fixed hyperparameters. The framework introduces no new theoretical entities. The main assumption is the sufficiency of the TSV abstraction, which is a domain assumption about data representation, not a mathematical axiom.

axioms (1)
  • domain assumption The TSV schema (signal, signal_start/end, encoder_prompt, decoder_prompt, output) and generic processors are expressive enough to represent any multimodal SLP experiment without task-specific code.
    This is the core design assumption introduced in §3.1 and used to claim general applicability, but it is only demonstrated on two tasks. There may be SLP data or tasks that do not fit this schema (e.g., multiple simultaneous inputs, non-text outputs), as the survey respondents themselves noted (§A.1).

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of MultimodalHugs: Enabling Sign Language Processing in Hugging Face." pith.science (2026). https://pith.science/paper/3PXS5Y5W

@misc{pith2026250909729,
  author       = {Pith},
  title        = {Pith review of: MultimodalHugs: Enabling Sign Language Processing in Hugging Face},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3PXS5Y5W}},
  note         = {Machine review of arXiv:2509.09729}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In recent years, sign language processing (SLP) has gained importance in the general field of Natural Language Processing. However, compared to research on spoken languages, SLP research is hindered by complex ad-hoc code, inadvertently leading to low reproducibility and unfair comparisons. Existing tools that are built for fast and reproducible experimentation, such as Hugging Face, are not flexible enough to seamlessly integrate sign language experiments. This view is confirmed by a survey we conducted among SLP researchers. To address these challenges, we introduce MultimodalHugs, a framework built on top of Hugging Face that enables more diverse data modalities and tasks, while inheriting the well-known advantages of the Hugging Face ecosystem. Even though sign languages are our primary focus, MultimodalHugs adds a layer of abstraction that makes it more widely applicable to other use cases that do not fit one of the standard templates of Hugging Face. We provide quantitative experiments to illustrate how MultimodalHugs can accommodate diverse modalities such as pose estimation data for sign languages, or pixel data for text characters.

Figures

Figures reproduced from arXiv: 2509.09729 by Amit Moryossef, Carlos Escolano, Gerard Sant, Mathias M\"uller, Rico Sennrich, Sarah Ebling, Zifan Jiang.

Figure 1
Figure 1. Figure 1: Illustration of the primary modalities used [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the MultimodalHugs workflow. The figure illustrates the four core steps: a) dataset standardization using TSV files (ASL: American Sign Language); b) setup of training configurations and modules; c) streamlined model training; and d) evaluation through automatic generation and metrics. specific modifications. For instance, numeric pose sequences or video frames cannot directly leverage tokenize… view at source ↗
Figure 3
Figure 3. Figure 3: Schematic overview of the MultimodalHugs processing pipeline, covering all experiments presented in both demonstrations. Sequences can come from different modalities, each processed by a dedicated processor to produce the appropriate input tensors for the model. The right side illustrates the translation training workflow, where text labels are also processed and provided as targets. multimodalhugs-generat… view at source ↗
Figure 4
Figure 4. Figure 4: Example training pipeline for a pose-based experiment in [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Multitask strategy proposed by Zhang et al. (2024). The figure illustrates an encoder-decoder SLT model trained on four pretraining tasks, all implemented as sequence-to-sequence learning within a shared architecture. Input prompts (red), sign frames (green), and target outputs (blue) are combined in task-specific ways: “sign lang” refers to the sign language name; “src lang/tgt lang” are source and target… view at source ↗
Figure 6
Figure 6. Figure 6: Envisioned future training pipeline for mixed-modality sequences in [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 10 canonical work pages · 1 internal anchor

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Sadam Al-Azani and El-Sayed M. El-Alfy. 2017. https://doi.org/10.1016/j.procs.2017.05.365 Using word embedding and ensemble learning for highly imbalanced data sentiment analysis in short arabic text . Procedia Computer Science, 109:359--366. 8th International Conference on Ambient Systems, Networks and Technologies, ANT-2017 and the 7th International Con...

    work page doi:10.1016/j.procs.2017.05.365 2017

  4. [4]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Miko aj Bi\' n...

    2022

  5. [5]

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. 2023. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv prepri...

    Pith/arXiv arXiv 2023

  6. [6]

    Murat Aydoğan and Ali Karci. 2020. https://doi.org/10.1016/j.physa.2019.123288 Improving the accuracy using pre-trained word embeddings on deep neural networks for turkish text classification . Physica A: Statistical Mechanics and its Applications, 541:123288

    arXiv 2020

  7. [7]

    Alessia Battisti, Emma van den Bold, Anne G \"o hring, Franz Holzknecht, and Sarah Ebling. 2024. https://aclanthology.org/2024.signlang-1.2/ Person identification from pose estimates in sign language . In Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages: Evaluation of Sign Language Resources, pages 1...

    2024

  8. [8]

    Anya Belz, Shubham Agarwal, Anastasia Shimorina, and Ehud Reiter. 2021. https://doi.org/10.18653/v1/2021.eacl-main.29 A systematic review of reproducibility research in natural language processing . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 381--393, Online. Associati...

    doi:10.18653/v1/2021.eacl-main.29 2021

  9. [9]

    Anya Belz and Craig Thomson. 2023. https://aclanthology.org/2023.humeval-1.4/ The 2023 R epro NLP shared task on reproducibility of evaluations in NLP : Overview and results . In Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems, pages 35--48, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria

    2023

  10. [10]

    Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, and Meredith Ringel Morris. 2019. https://doi.org/10.1145/3308561.3353774 Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective . In The 21st ...

    arXiv 2019

  11. [11]

    Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. 2020. Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  12. [12]

    Necati Cihan Camgöz, Ben Saunders, Guillaume Rochette, Marco Giovanelli, Giacomo Inches, Robin Nachtrab-Ribback, and Richard Bowden. 2021. https://doi.org/10.1109/FG52635.2021.9667087 Content4all open research sign language translation datasets . In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), pages 1--5

    arXiv 2021

  13. [13]

    Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. https://doi.org/10.1109/TPAMI.2019.2929257 Openpose: Realtime multi-person 2d pose estimation using part affinity fields . IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1):172--186

    arXiv 2021

  14. [14]

    Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2017

  15. [15]

    Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie LIU, and Brian Mak. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/6cd3ac24cdb789beeaa9f7145670fcae-Paper-Conference.pdf Two-stream network for sign language recognition and translation . In Advances in Neural Information Processing Systems, volume 35, pages 17043--17056. Curran Associ...

    2022

  16. [16]

    Avihay Chriqui and Inbal Yahav. 2022. https://doi.org/10.1287/ijds.2022.0016 Hebert and hebemo: A hebrew bert model and a tool for polarity analysis and emotion recognition . INFORMS Journal on Data Science, 1(1):81--95

    arXiv 2022

  17. [17]

    Mathieu De Coster, Karel D ' Oosterlinck, Marija Pizurica, Paloma Rabaey, Severine Verlinden, Mieke Van Herreweghe, and Joni Dambre. 2021. https://aclanthology.org/2021.mtsummit-at4ssl.10/ Frozen pretrained transformers for neural sign language translation . In Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken La...

    2021

  18. [18]

    Jesse Dodge. 2020. Towards Efficient and Reproducible Natural Language Processing. Ph.D. thesis, Carnegie Mellon University

    2020

  19. [19]

    Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro-i Nieto. 2021. How2sign: A large-scale multimodal dataset for continuous american sign language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2735--2744

    2021

  20. [20]

    Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Michael Auli, and Armand Joulin. 2021. http://jmlr.org/papers/v22/20-1307.html Beyond english-centric multilingual machine translation . Journa...

    2021

  21. [21]

    Hossam Faris et al. 2020. https://www.scitepress.org/Papers/2020/89540/89540.pdf Hate speech detection using word embedding and deep learning in the arabic language context . In ICPRAM, pages 453--460

    2020

  22. [22]

    Awet Fesseha, Shengwu Xiong, Eshete Derb Emiru, Moussa Diallo, and Abdelghani Dahou. 2021. https://doi.org/10.3390/info12020052 Text classification based on convolutional neural networks and word embedding for low-resource languages: Tigrinya . Information, 12(2)

    work page doi:10.3390/info12020052 2021

  23. [23]

    Mohammed M Fouad, Ahmed Mahany, Naif Aljohani, Rabeeh Ayaz Abbasi, and Saeed-Ul Hassan. 2020. https://link.springer.com/article/10.1007/s00500-019-04153-6 Arwordvec: efficient word embedding models for arabic tweets . Soft Computing, 24(11):8061--8068

    work page doi:10.1007/s00500-019-04153-6 2020

  24. [24]

    Ivan Grishchenko and Valentin Bazarevsky. 2020. https://ai.googleblog.com/2020/12/mediapipe-holistic-simultaneous-face.html MediaPipe Holistic - simultaneous face, hand and pose prediction, on device . Google AI Blog

    2020

  25. [25]

    Jelinek, R

    F. Jelinek, R. L. Mercer, L. R. Bahl, and J. K. Baker. 2005. https://doi.org/10.1121/1.2016299 Perplexity—a measure of the difficulty of speech recognition tasks . The Journal of the Acoustical Society of America, 62(S1):S63--S63

    doi:10.1121/1.2016299 2005

  26. [26]

    Abhinav Joshi, Romit Mohanty, Mounika Kanakanti, Andesha Mangla, Sudeep Choudhary, Monali Barbate, and Ashutosh Modi. 2024. https://doi.org/10.18653/v1/2024.findings-acl.643 i S ign: A benchmark for I ndian S ign L anguage processing . In Findings of the Association for Computational Linguistics: ACL 2024, pages 10827--10844, Bangkok, Thailand. Associatio...

    work page doi:10.18653/v1/2024.findings-acl.643 2024

  27. [27]

    Tarek Kanan, Odai Sadaqa, Amal Aldajeh, Hanadi Alshwabka, Wassan AL-dolime, Shadi AlZu’bi, Mohammed Elbes, Bilal Hawashin, and Mohammad A. Alia. 2019. https://doi.org/10.1109/JEEIT.2019.8717369 A review of natural language processing and machine learning tools used to analyze arabic social media . In 2019 IEEE Jordan International Joint Conference on Elec...

    arXiv 2019

  28. [28]

    Yin Kayo and Read Jesse. 2020. https://doi.org/10.18653/v1/2020.coling-main.525 Better sign language translation with stmc-transformer . Proceedings of the 28th International Conference on Computational Linguistics

    doi:10.18653/v1/2020.coling-main.525 2020

  29. [29]

    Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. https://aclanthology.org/P17-4012/ O pen NMT : Open-source toolkit for neural machine translation . In Proceedings of ACL 2017, System Demonstrations , pages 67--72, Vancouver, Canada. Association for Computational Linguistics

    2017

  30. [30]

    Stav Klein and Reut Tsarfaty. 2020. https://doi.org/10.18653/v1/2020.sigmorphon-1.24 Getting the \# \# life out of living: How adequate are word-pieces for modelling complex morphology? In Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 204--209, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.sigmorphon-1.24 2020

  31. [31]

    Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/d49042a5d49818711c401d34172f9900-Paper-Datasets_and_Benchmarks.pdf Madlad-400: A multilingual and document-level large audited dataset . In Advances in Neural I...

    2023

  32. [32]

    Elmurod Kuriyozov, Yerai Doval, and Carlos G \'o mez-Rodr \'i guez. 2020. https://aclanthology.org/2020.lrec-1.499/ Cross-lingual word embeddings for T urkic languages . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4054--4062, Marseille, France. European Language Resources Association

    2020

  33. [33]

    Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, and Jimmy Huang. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.764 A systematic survey and critical review on evaluating large language models: Chall...

    doi:10.18653/v1/2024.emnlp-main.764 2024

  34. [34]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/6dcf277ea32ce3288914faf369fe6de0-Paper-Conference.pdf Visual instruction tuning . In Advances in Neural Information Processing Systems, volume 36, pages 34892--34916. Curran Associates, Inc

    2023

  35. [35]

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. 2019. https://arxiv.org/abs/1906.08172 Mediapipe: A framework for building perception pipelines . Preprint, arXiv:1906.08172

    Pith/arXiv arXiv 2019

  36. [36]

    Amit Moryossef, Mathias M \" u ller, and Rebecka Fahrni. 2023. https://doi.org/10.48550/ARXIV.2310.09066 pose-format: Library for viewing, augmenting, and handling .pose files . CoRR, abs/2310.09066

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2310.09066 2023

  37. [37]

    u ller, Sarah Ebling, Eleftherios Avramidis, Alessia Battisti, Mich \`e le Berger, Richard Bowden, Annelies Braffort, Necati Cihan Camg \

    Mathias M \"u ller, Sarah Ebling, Eleftherios Avramidis, Alessia Battisti, Mich \`e le Berger, Richard Bowden, Annelies Braffort, Necati Cihan Camg \"o z, Cristina Espa \ n a-bonet, Roman Grundkiewicz, Zifan Jiang, Oscar Koller, Amit Moryossef, Regula Perrollaz, Sabine Reinhard, Annette Rios, Dimitar Shterionov, Sandra Sidler-miserez, and Katja Tissi. 202...

    2022

  38. [38]

    Mathias M \"u ller, Zifan Jiang, Amit Moryossef, Annette Rios, and Sarah Ebling. 2023 a . https://doi.org/10.18653/v1/2023.acl-short.60 Considerations for meaningful sign language machine translation based on glosses . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 682--693, Toron...

    work page doi:10.18653/v1/2023.acl-short.60 2023

  39. [39]

    Mathias M \"u ller, Annette Rios, Amit Moryossef, Sarah Ebling, and Thomas Hanke UHH. 2023 b . https://www.project-easier.eu/wp-content/uploads/sites/67/2023/08/EASIER_D4.3_Translation-models-final-version-v.1.2.pdf D4.3 easier final translation systems v2

    2023

  40. [40]

    Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. https://doi.org/10.18653/v1/N19-4009 fairseq: A fast, extensible toolkit for sequence modeling . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics (Demonstrations) , pages 48...

    doi:10.18653/v1/n19-4009 2019

  41. [41]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics

    arXiv 2002

  42. [42]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395, Lisbon, Portugal. Association for Computational Linguistics

    doi:10.18653/v1/w15-3049 2015

  43. [43]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. https://proceedings.mlr.press/v139/radford21a.html Learning transferable visual models from natural language supervision . In Proceedings of the 38th International C...

    2021

  44. [44]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.213 COMET : A neural framework for MT evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685--2702, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.213 2020

  45. [45]

    Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott

    Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott. 2023. https://openreview.net/forum?id=FkSp8VW8RjH Language modelling with pixels . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net

    2023

  46. [46]

    Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, and Armand Joulin. 2020. https://arxiv.org/abs/1911.04944 Ccmatrix: Mining billions of high-quality parallel sentences on the web . Preprint, arXiv:1911.04944

    Pith/arXiv arXiv 2020

  47. [47]

    Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. 2022. https://doi.org/10.18653/v1/2022.acl-long.4 A leph BERT : Language model pre-training and evaluation from sub-word to sentence level . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

    work page doi:10.18653/v1/2022.acl-long.4 2022

  48. [48]

    Telem Joyson Singh, Sanasam Ranbir Singh, and Priyankoo Sarmah. 2023. https://aclanthology.org/2023.paclic-1.69 Subwords to word back composition for morphologically rich languages in neural machine translation . In Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation, pages 691--700, Hong Kong, China. Association for C...

    2023

  49. [49]

    Valerie Sutton. 1990. Lessons in sign writing. SignWriting

    1990

  50. [50]

    S. Tan, N. Khan, Z. An, Y. Ando, and R. Kawakami. 2024. https://doi.org/10.1080/01691864.2024.2442721 A review of deep learning-based approaches to sign language processing . Advanced Robotics, 38(23):1649--1667(19)

    arXiv 2024

  51. [51]

    Garrett Tanzer and Biao Zhang. 2024. https://arxiv.org/abs/2407.11144 Youtube-sl-25: A large-scale, open-domain multilingual sign language parallel corpus . Preprint, arXiv:2407.11144

    Pith/arXiv arXiv 2024

  52. [52]

    G\'allego, Amanda Duarte, Jordi Torres, and Xavier Gir\'o-i Nieto

    Laia Tarr\'es, Gerard I. G\'allego, Amanda Duarte, Jordi Torres, and Xavier Gir\'o-i Nieto. 2023. https://openaccess.thecvf.com/content/CVPR2023W/WiCV/html/Tarres_Sign_Language_Translation_from_Instructional_Videos_CVPRW_2023_paper.html Sign language translation from instructional videos . In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2023

  53. [53]

    NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe,...

    Pith/arXiv arXiv 2022

  54. [54]

    Yingli Tian, Jianbo Su, Lan Ni, and Yuchun Fang. 2024. https://doi.org/10.1142/S2972335324010038 Bridging the gap: Ai and sign language recognition– a path toward inclusive communication . International Journal of Artificial Intelligence and Robotics Research, 0(0):2401003

    work page doi:10.1142/s2972335324010038 2024

  55. [55]

    Reut Tsarfaty, Djam \'e Seddah, Yoav Goldberg, Sandra Kuebler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein, and Lamia Tounsi. 2010. https://aclanthology.org/W10-1401/ Statistical parsing of morphologically rich languages ( SPMRL ) what, how and whither . In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morpho...

    2010

  56. [56]

    Dave Uthus, Garrett Tanzer, and Manfred Georg. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/5c61452daca5f0c260e683b317d13a3f-Paper-Datasets_and_Benchmarks.pdf Youtube-asl: A large-scale, open-domain american sign language-english parallel corpus . In Advances in Neural Information Processing Systems, volume 36, pages 29029--29047. Curr...

    2023

  57. [57]

    Leandro Von Werra, Lewis Tunstall, Abhishek Thakur, Sasha Luccioni, Tristan Thrush, Aleksandra Piktus, Felix Marty, Nazneen Rajani, Victor Mustar, and Helen Ngo. 2022. https://doi.org/10.18653/v1/2022.emnlp-demos.13 Evaluate & evaluation on the hub: Better best practices for data and model measurements . In Proceedings of the 2022 Conference on Empirical ...

    work page doi:10.18653/v1/2022.emnlp-demos.13 2022

  58. [58]

    Waghmare and Ashwini M

    Prachi P. Waghmare and Ashwini M. Deshpande. 2023. https://ijritcc.org/index.php/ijritcc/article/view/9765/7458 A study on techniques and challenges in sign language translation . International Journal on Recent and Innovation Trends in Computing and Communication, 11(9):4039--4050. ISSN: 2321-8169

    2023

  59. [59]

    Philip Whittington, Gregor Bachmann, and Tiago Pimentel. 2024. https://arxiv.org/abs/2412.15210 Tokenisation is np-complete . Preprint, arXiv:2412.15210

    Pith/arXiv arXiv 2024

  60. [60]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. https://doi.org/10.18653...

    doi:10.18653/v1/2020.emnlp-demos.6 2020

  61. [61]

    Linting Xue, Aditya Barua, Noah Constant, Rami Al - Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. https://doi.org/10.1162/TACL\_A\_00461 Byt5: Towards a token-free future with pre-trained byte-to-byte models . Trans. Assoc. Comput. Linguistics, 10:291--306

    doi:10.1162/tacl 2022

  62. [62]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. https://arxiv.org/abs/2010.11934 mt5: A massively multilingual pre-trained text-to-text transformer . Preprint, arXiv:2010.11934

    Pith/arXiv arXiv 2021

  63. [63]

    Kayo Yin, Amit Moryossef, Julie Hochgesang, Yoav Goldberg, and Malihe Alikhani. 2021. https://doi.org/10.18653/v1/2021.acl-long.570 Including signed languages in natural language processing . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing...

    work page doi:10.18653/v1/2021.acl-long.570 2021

  64. [64]

    Biao Zhang, Garrett Tanzer, and Orhan Firat. 2024. https://arxiv.org/abs/2407.11855 Scaling sign language translation . Preprint, arXiv:2407.11855

    Pith/arXiv arXiv 2024

  65. [65]

    Ce Zheng, Wenhan Wu, Chen Chen, Taojiannan Yang, Sijie Zhu, Ju Shen, Nasser Kehtarnavaz, and Mubarak Shah. 2023. https://doi.org/10.1145/3603618 Deep learning-based human pose estimation: A survey . ACM Comput. Surv., 56(1)

    doi:10.1145/3603618 2023

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.