Sign Spotting Disambiguation using Large Language Models

JianHe Low; Ozge Mercanoglu Sincan; Richard Bowden

REVIEW 4 major objections 5 minor 37 references

Sign Spotting Disambiguation using Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A frozen large language model, used as a beam-search scorer, lowers word error in dictionary-based sign spotting by re-ranking visually plausible gloss sequences.

desk verdict A plausible training-free LLM reranker for sign spotting, but the evidence for the headline gain rests on a small internal dataset and a synthetic noise model that seems to inject the wrong kind of errors. read the letter →

arxiv 2507.03703 v4 pith:JW5RHDAO submitted 2025-07-04 cs.CV cs.AI

JianHe Low , Ozge Mercanoglu Sincan , Richard Bowden This is my paper

classification cs.CVcs.AI

keywords signlanguagespottingglossdisambiguationlargemodelsbeamsearchdecodingdictionary-basedmatchingdynamictimewarpingworderrorratetraining-freeannotation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sign spotting—naming and locating individual signs inside continuous signing—could make large-scale sign language annotation affordable, but isolated sign predictions are often ambiguous because visually similar signs carry different meanings. This paper tries to establish that a frozen large language model (LLM) can resolve that ambiguity by scoring whole candidate gloss sequences instead of single signs. The visual dictionary matcher proposes a top-k set of gloss candidates for each spotted segment, and the LLM's next-token probabilities act as transition scores in a beam search that combines them with visual similarity scores. The method needs no retraining of the visual encoder and no fine-tuning of the LLM. On real continuous signing paired with a 1,000-word dictionary, the authors report Top-1 word error rates of 44.73% and 44.38% for the two LLMs, with a best Top-5 WER of 34.81%, down from 47.24% for the visual-only late-fusion matcher.

What carries the argument

The load-bearing mechanism is a beam-search decoder with LLM-supplied transition probabilities. Each spotted segment contributes a candidate set $C_x=\{(g^j_x,s^j_x)\}_{j=1}^{k}$ of top-$k$ glosses with visual emission scores $s^j_x$; the frozen LLM, prompted with the previous glosses and the candidate options, supplies $p(g_x\mid g_{1:x-1})$ as the next-token probability of each candidate. The decoder solves $\hat{g}_{1:X}=\arg\max_{g_{1:X}\in\prod_x C_x}\sum_x(\log p(g_x\mid g_{1:x-1})+\alpha_{\mathrm{bs}}s_x)$ with beam width 5, so linguistic fluency and visual evidence are traded off by a single hyperparameter $\alpha_{\mathrm{bs}}$. Around this sits the training-free dictionary matcher: I3D spatiotemporal embeddings and ResNeXt-101 hand-shape embeddings are matched to separate dictionaries with dynamic time warping plus cosine similarity, and late/intermediate/ensemble fusion builds the candidate distributions that the LLM then re-ranks.

What would settle it

Rerun the full-system evaluation on a public continuous sign dataset paired with an isolated-sign dictionary, preserving the same late-fusion matcher and frozen LLM; if Top-1 WER does not fall when the LLM is added, or if every LLM-corrected gloss is absent from the visual top-$k$ list, the reported gain is not a general property of the mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that adding a frozen large language model as a context-aware scorer to a dictionary-based sign spotter reduces word error, because the matcher's top-k candidates usually contain the right gloss but rank it below visually similar alternatives. The LLM is prompted with the previously selected glosses and asked which candidate completes the sequence; its logits become transition probabilities, and a beam search combines them with visual similarity scores. In the authors' experiments this lowers real-video Top-1 WER from 0.4724 to 0.4438–0.4473 depending on the LLM, with the best Top-5 WER at 0.3481. Synthetic evaluations under heavy noise confirm the mechanism: the LLM can correct even fully wrong top-1 choices whenever the true gloss remains inside the candidate set.

Load-bearing premise

The argument stands on the Section 4.1 synthetic evaluation: the injected word-replacement and distribution-corruption noise is assumed to mimic the real dictionary matcher's error pattern closely enough that hyperparameters tuned on it transfer to continuous video, where the claimed real-data gain is measured once on a single internal dataset.

Editorial extensions

If this is right

New sign vocabulary can be added to the spotter by inserting new dictionary entries, and the frozen LLM scorer applies without any visual retraining, so the pipeline scales to datasets whose vocabularies are not fixed in advance.
The LLM corrects visually plausible but linguistically wrong glosses (for example 'US' to 'WE-ALL' and 'SHORT' to 'MAKE' in the paper's qualitative examples), which makes spotter output closer to readable translation even when individual signs were misranked.
Because beam search keeps several hypotheses, the correct sequence can survive at a lower rank when the top choice is wrong, explaining the consistent gap between Top-1 and Top-5 WER.
Across the synthetic noise sweep, the larger 9B Gemma-2 model outperformed the smaller Phi-3 Mini, indicating the linguistic prior improves with model capacity.
Spotting quality is bounded by the visual candidate list: under extreme synthetic noise, neither LLM can recover the target gloss once it is pushed out of the top-$k$ candidates, so the visual matcher's recall and the beam width set the ceiling for the disambiguation module.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct transfer test would be to pair a public continuous sign dataset with an isolated-sign dictionary and rerun the same late-fusion matcher with and without the frozen LLM; the paper notes that such paired public datasets are scarce, so the real-video result is reported on one internal dataset.
The synthetic ablations suggest that improving visual recall may yield larger gains than scaling the LLM, since the disambiguator can only rerank what the matcher puts in its candidate set; the paper does not quantify that trade-off.
The same pattern—using a frozen language model's transition scores to rerank a retrieval list—should transfer to other n-best post-processing tasks such as keyword spotting in archival video or OCR/ASR correction, wherever linguistic coherence matters.
The large Top-5/Top-1 gap implies the correct gloss is often the matcher's second or third choice; an annotation interface that shows the top-ranked alternatives could let a human validator accept the right sequence quickly, a practical consequence the paper leaves implicit.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The new thing here is the recipe: put LLM next-token probabilities into a beam search as transition scores, on top of dictionary-based sign spotting with DTW/cosine retrieval. That integration is clean, training-free, and I don't see it in the prior work they cite, including their own [25], which uses an LLM differently. The paper also does a decent job laying out fusion variants and ablating them. If you need a lightweight, vocabulary-flexible spotter, the dictionary-plus-LLM structure is worth knowing about.

The soft spot is the evaluation, and it's load-bearing. The real dataset is internal, unreleased, and the reported gain is small: Late Fusion WER goes 0.4724 → 0.4438 with Phi-3, and Gemma-2 is no better (0.4473). No error bars, no significance test, no non-LLM language model baseline (a simple n-gram would be the obvious control). The conclusion calls this "significant," which is overstatement.

The bigger problem is the synthetic noise model in Section 4.1.3, used for all hyperparameter tuning. Word Replacement injects a random word at the top; Distribution Corruption boosts semantically dissimilar words. But the real confusions shown in Table 7 are semantically and visually close (ME/I/MYSELF, TRAIN/RAIL). An LLM's grammatical priors can reject a random word, but they can't easily pick between "ME" and "I" when both are grammatically fine. So the HPO is tuned against an error structure that may not match real matcher errors, and the small real gain, plus the lack of a size effect between Phi-3 and Gemma-2, is consistent with the LLM not resolving the actually-occurring ambiguities.

That said, the stress-test note may overstate the case: the synthetic pipeline also includes some realistic FastText-based similarity, and the qualitative examples show the LLM did fix "US" to "WE-ALL" and "SHORT" to "MAKE" on real data. So the idea isn't dead, just unproven.

Who is this for? Researchers in sign spotting or annotation scaling who want a plug-in disambiguation module. It deserves a serious referee, but only with major revision: release the data or run on a public benchmark, add an n-gram baseline, report variance, and redesign the synthetic noise to reflect semantically similar confusions. As it stands, the central claim isn't established.

Referee Report

4 major / 5 minor

Summary. The paper proposes a training-free framework for sign spotting that combines dictionary-based visual matching with a frozen large language model (LLM) used as a contextual scorer in beam search. The visual front-end extracts I3D and hand-shape features, matches them against a dictionary of isolated sign videos using a weighted combination of DTW and cosine similarity, and optionally fuses I3D and hand-stream similarity distributions. The LLM provides transition probabilities p(gloss|previous glosses), which are combined with visual emission scores in the beam search objective of Eq. (6). The authors evaluate the module in isolation on synthetic pseudo-gloss data with injected noise (word replacement and distribution corruption), tune the fusion and beam-search hyperparameters on that synthetic data, and then report real-video results on an internally collected continuous sign language dataset with a 1,000-entry dictionary. The headline real result is a Top-1 WER reduction from 0.4724 (late fusion without disambiguation) to 0.4438 with Phi-3 and 0.4473 with Gemma-2 (Table 6). The paper claims that this integration substantially reduces word error rate and demonstrates the potential of LLMs in sign spotting.

Significance. If the real-video gains were robust, the paper would offer a modular and practical contribution: it combines a flexible dictionary-based spotter with a frozen, publicly available LLM and avoids retraining for new vocabulary, which matters for dataset annotation at scale. The design is simple, the equations in Section 3 are internally consistent, and the synthetic evaluation in Section 4.1 is a reasonable first step for ablating the decoder in isolation. The main value of the idea, however, is not yet established by the evidence presented. The real-data improvement is small (roughly 6% relative WER reduction), unreplicated, and reported without uncertainty measures; the synthetic noise model used for hyperparameter selection and for the ablation story injects errors with a different semantic structure than the real confusions documented in Section 4.3.1; and there is no non-LLM language-model baseline that would isolate the effect of the linguistic prior. These are load-bearing gaps for the paper's central claim that LLM-based disambiguation significantly enhances sign spotting quality.

major comments (4)

[4.1.3] The fidelity of the synthetic noise model is load-bearing, because Tables 1-4 are used to select alpha_s, alpha_late, alpha_ens, alpha_bs, and beam width, and Table 3 is used to argue that the LLM has 'clear disambiguation capabilities'. The Word Replacement and Distribution Corruption injections replace the top-1 gloss with a random word or boost semantically dissimilar words, but the real confusions shown in Section 4.3.1 (ME/I/MYSELF, ME/MY, TRAIN/RAIL) are visually similar and semantically related. An LLM can reject a randomly inserted word using strong grammatical priors, whereas it cannot easily resolve errors that are already linguistically plausible. The authors should validate that the synthetic error distribution matches the real matcher's candidate distribution, for example by fitting WR/DC parameters to held-out real similarity distributions or by directly measuring the semantic relation between visual top-k confusions and ground truth.
[4.2.2] The real-data claim rests on small, unreplicated differences. In Table 6, late fusion WER moves from 0.4724 to 0.4438 with Phi-3 and to 0.4473 with Gemma-2; the full-ensemble row moves from 0.4924 to 0.4567. The paper does not report the number of videos, sign units, signers, or evaluation segments in the internal dataset, nor does it give confidence intervals, bootstrap estimates, or any significance test. The differences are small enough that they could plausibly result from segment-boundary choices or run-to-run variation in feature extraction. The authors should report dataset statistics, repeat the evaluation or compute bootstrapped intervals, and state whether the observed differences are statistically reliable.
[3.2.3] The claim that the LLM specifically provides the benefit is not isolated from the effect of any language model. Equation (6) only uses a generic transition probability p(gloss_x | gloss_1:x-1), so a trigram model or a small neural language model trained on text would be a natural lower-cost baseline. Without such a baseline, the experimental setup cannot distinguish 'linguistic context helps' from 'LLM-scale linguistic priors help', which is the paper's stated contribution.
[4.2] The evaluation dataset is described only as 'an internally collected continuous sign language dataset' paired with a 1,000-entry dictionary. No details are given about the domain, number of signers, recording conditions, gloss annotation protocol, or dictionary construction for the real experiment. This makes the reported WER numbers difficult to interpret and impossible to reproduce or compare against future work. The authors should include at least the dataset size, segmentation procedure, vocabulary composition, and dictionary source for the real evaluation, or release the data.

minor comments (5)

[3.2.3] The notation in Eq. (6) uses a product symbol rendered as 'Î' over candidate sets; this should be a standard Cartesian-product symbol, and the sentence 'The hyperparameter alpha_bs weighs the the LLM's linguistic score' contains a duplicated 'the'.
[4.2.2] The metric 'Top-5 WER' is never defined. It appears to mean the WER of the sequence with the lowest WER among the beam's top five hypotheses, but the paper should state this explicitly and, ideally, also report oracle Top-k WER for the visual-only baseline to put the Top-5 numbers in context.
[4.1.1] The text cites BSLDict with reference [18], but reference [18] is Momeni et al. 'Watch, read and lookup' while the BSLDict dictionary work appears to be reference [30] (Varol et al., 'Scaling up sign spotting through sign language dictionaries'). The citation should be corrected.
[4.3.1] In Table 7, the second row of the Top-3 prediction contains 'W ALKING' with an inserted space, and the phrase 'FS_WOOD' versus 'JUNE' in the second and third hypotheses is not explained; a brief note on fingerspelling tokens would help readability.
[1] The term 'training-free' is used for the overall framework, but the feature extractors are pretrained and the fusion weights are tuned. The authors should clarify that 'training-free' means no fine-tuning of the LLM and no retraining for new dictionary entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the LLM and visual matcher are independent components, hyperparameters are tuned on a synthetic proxy rather than the held-out test set, and the one overlapping prior work only supplies a feature extractor and baseline.

full rationale

The paper's central derivation is not circular by construction. Emission probabilities in Eq. 6 come from DTW/cosine dictionary matching against a 1000-entry sign dictionary (Sec. 3.1.2), and transition probabilities come from frozen, publicly released LLMs (Phi-3 Mini and Gemma-2 9B) accessed through next-token logits (Sec. 3.2.3). Neither probability source is defined in terms of the other, so the beam-search objective is a genuine combination of independent visual and linguistic signals. The hyperparameters αs, αlate, αens, and beam width are selected on synthetic data (Sec. 4.1.4, Tables 1-4) and then applied to a separate real continuous-sign dataset (Sec. 4.2, Tables 5-6); they are not fitted to the real test outputs, so the real WER comparisons are not self-predictions. The only author-overlapping citation used as a system component is [25], which supplies the pretrained/finetuned I3D encoder and serves as a baseline; this is an external feature extractor and does not justify the disambiguation claim, so it is not load-bearing circularity. The synthetic evaluation is best viewed as a proxy-validity risk rather than a circular step: the pseudo-glosses are derived from English sentences and the injected WR/DC noise is random, so the synthetic results may overstate how well the LLM can resolve real, visually similar confusions (e.g., ME/I/MYSELF in Sec. 4.3.1). However, the paper explicitly labels this as a simulation, and the real-data experiments provide an independent check, so no equation or fitted parameter is equivalent to the claimed output by definition.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities. It does introduce several fitted hyperparameters (alphas, beam width, candidate count) and relies on strong domain assumptions about English-to-BSL transferability, LLM gloss-token competence, and the fidelity of synthetic noise. The largest burden is the assumption that synthetic English-based pseudo-glosses simulate real visual matcher errors well enough to tune the system.

free parameters (6)

alpha_s = I3D: ~0.2-0.3 (Top-5), RH: 0.9 (Top-1)
Weight between DTW and cosine similarity in Eq. 3, tuned on synthetic data (Table 1).
alpha_late = 0.9 (Top-5 optimal)
Weight between I3D and RH similarity distributions in Eq. 4, tuned on synthetic data (Table 1).
alpha_ens = 0.6
Weight between intermediate fusion and I3D distributions in Eq. 5, tuned on synthetic data (Table 1).
alpha_bs = not reported
Weight of LLM transition probability vs emission probability in the beam search objective (Eq. 6); value is missing from the paper.
top-k = not reported
Number of gloss candidates passed to the LLM; affects the candidate search space and is not specified for the main experiments.
beam_width = 5 (main experiments); up to 50 in ablations
Beam search width in Eq. 6; chosen as 5 in Section 3.2.3, with ablations in Table 4.

assumptions (5)

domain assumption Pretrained I3D (BOBSL-finetuned) and ResNeXt-101 (1M hands) extract reliable visual features for sign matching.
Invoked in Section 3.1.1 without evaluation of feature quality on the internal dataset.
ad hoc to paper English word frequency lists and POS-filtered English sentences produce realistic pseudo-gloss sequences.
Synthetic dictionary and pseudo-gloss generation in Sections 4.1.1-4.1.2 assume English text is a valid proxy for BSL gloss sequences.
domain assumption LLM next-token probabilities over English gloss tokens are a valid linguistic prior for sign language gloss sequencing.
Core to the disambiguation module in Sections 3.2.2-3.2.3; no validation that the LLM handles non-lexical glosses (e.g., FS_CHRIS, WE-ALL) well.
domain assumption The 1,000-entry isolated sign dictionary contains representative templates for matching continuous signing segments.
Dictionary construction in Section 3.1.2 assumes isolated samples capture the visual variability of continuous signing.
standard math DTW and cosine similarity on concatenated features are adequate similarity measures for gloss retrieval.
Used in Eq. 3 without comparison to alternative retrieval or alignment methods.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Sign Spotting Disambiguation using Large Language Models." pith.science (2026). https://pith.science/paper/JW5RHDAO

@misc{pith2026250703703,
  author       = {Pith},
  title        = {Pith review of: Sign Spotting Disambiguation using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JW5RHDAO}},
  note         = {Machine review of arXiv:2507.03703}
}

read the original abstract

Sign spotting, the task of identifying and localizing individual signs within continuous sign language video, plays a pivotal role in scaling dataset annotations and addressing the severe data scarcity issue in sign language translation. While automatic sign spotting holds great promise for enabling frame-level supervision at scale, it grapples with challenges such as vocabulary inflexibility and ambiguity inherent in continuous sign streams. Hence, we introduce a novel, training-free framework that integrates Large Language Models (LLMs) to significantly enhance sign spotting quality. Our approach extracts global spatio-temporal and hand shape features, which are then matched against a large-scale sign dictionary using dynamic time warping and cosine similarity. This dictionary-based matching inherently offers superior vocabulary flexibility without requiring model retraining. To mitigate noise and ambiguity from the matching process, an LLM performs context-aware gloss disambiguation via beam search, notably without fine-tuning. Extensive experiments on both synthetic and real-world sign language datasets demonstrate our method's superior accuracy and sentence fluency compared to traditional approaches, highlighting the potential of LLMs in advancing sign spotting.

Figures

Figures reproduced from arXiv: 2507.03703 by the authors.

**Figure 2.** Overall Architecture. Our approach features two main stages: (a) Sign Spotting: Sign segments are first obtained from [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Illustration of Dictionary Look-Up and Late Fusion. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Illustration of Mid Fusion and Full-Ensemble. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Visualization of the Disambiguation Process. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 6.** Figure 6: Example of Our Prompt Template. 3.2.2 Prompt Formulation. The prompt provided to the LLM is dynamically constructed to incorporate both contextual information and candidate glosses. As seen in [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 23 canonical work pages

[25]

Ozge Mercanoglu Sincan, Necati Cihan Camgoz, and Richard Bowden. 2024. Using an LLM to turn sign spottings into spoken language sentences. arXiv preprint arXiv:2403.10434 (2024)

arXiv 2024
[1]

Samuel Albanie, Gül Varol, Liliane Momeni, Hannah Bull, Triantafyllos Afouras, Himel Chowdhury, Neil Fox, Bencie Woll, Rob Cooper, and Andrew Zisserman McParland. 2021. BBC-Oxford British Sign Language Dataset. arXiv preprint arXiv:2111.03635 (2021)

arXiv 2021
[2]

Sally Beatty. 2024. Tiny but Mighty: The Phi-3 Small Language Models with Big Potential. https://news.microsoft.com/source/features/ai/the-phi-3-small- language-models-with-big-potential/. Accessed: 6-6-2025

work page 2024
[3]

Berndt and James Clifford

Donald J. Berndt and James Clifford. 1994. Using Dynamic Time Warping to Find Patterns in Time Series. In Proceedings of the 3rd international conference on knowledge discovery and data mining

work page 1994
[4]

Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. 2018. Neural Sign Language Translation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 7784–7793

work page 2018
[5]

Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 6299–6308

2017
[6]

Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, and Stephen Lin. 2022. A simple multi-modality transfer learning baseline for sign language translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5120–5130

2022
[7]

Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, and Brian Mak. 2022. Two-stream network for sign language recognition and translation. Advances in Neural Information Processing Systems 35 (2022), 17043–17056

2022

Show all 37 references

[8]

Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro-i Nieto. 2021. How2sign: a large-scale multimodal dataset for continuous american sign language. In Proceedings of the IEEE/CVF conference on computer...

2021
[9]

Jordan Fenlon, Adam Schembri, Ramas Rentelis, David Vinson, and Kearsy Cormier. 2014. Using conversational data to determine lexical frequency in British Sign Language: The influence of text type. Lingua 143 (2014), 187–202

2014
[10]

Low Jian He, Harry Walsh, Ozge Mercanoglu Sincan, and Richard Bowden. 2025. Hands-On: Segmenting Individual Signs from Continuous Sequences. In 2025 IEEE 19th International Conference on Automatic Face and Gesture Recognition (FG). 1–5. doi:10.1109/FG61629.2025.11099255

2025
[11]

Lianyu Hu, Wei Feng, Liqing Gao, Zekang Liu, and Liang Wan. 2024. Corrnet+: Sign language recognition and translation via spatial-temporal correlation. arXiv preprint arXiv:2404.11111 (2024)

2024 arXiv
[12]

Tao Jiang, Necati Cihan Camgöz, and Richard Bowden. 2021. Looking for the signs: Identifying isolated sign instances in continuous video footage. In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021). IEEE, 1–8

2021
[13]

Oscar Koller, Hermann Ney, and Richard Bowden. 2016. Deep hand: How to train a cnn on 1 million hand images when your data is continuous and weakly labelled. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3793–3802

2016
[14]

Reiner Konrad, Thomas Hanke, Gabriele Langer, Dolly Blanck, Julian Bleicken, Ilona Hofmann, Olga Jeziorski, Lutz König, Susanne König, Rie Nishio, et al. 2020. MEINE DGS – annotiert. Öffentliches Korpus der Deutschen Gebärdensprache, 3. Release / MY DGS – Annotated. Public Cor...

2020
[15]

Zecheng Li, Wengang Zhou, Weichao Zhao, Kepeng Wu, Hezhen Hu, and Houqiang Li. 2025. Uni-sign: Toward unified sign language understanding at scale. arXiv preprint arXiv:2501.15187 (2025)

2025 arXiv
[16]

JianHe Low, Ozge Mercanoglu Sincan, and Richard Bowden. 2025. SAGE: Segment-Aware Gloss-Free Encoding for Token-Efficient Sign Language Transla- tion. arXiv preprint arXiv:2507.09266 (2025)

2025 arXiv
[17]

Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, et al. 2019. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019)

2019 arXiv
[18]

Liliane Momeni, Gul Varol, Samuel Albanie, Triantafyllos Afouras, and Andrew Zisserman. 2020. Watch, read and lookup: learning to spot signs from multiple supervisors. In Proceedings of the Asian Conference on Computer Vision

2020
[19]

Meinard Müller. 2007. Information retrieval for music and motion . Springer

2007
[20]

National Deaf Children’s Society. 2024. What is Sign Language? https://www.ndcs.org.uk/information-and-support/language-and- communication/sign-language/what-is-sign-language/. Accessed: 22.03.2024

2024
[21]

Adrián Núñez-Marcos, Olatz Perez-de Viñaspre, and Gorka Labaka. 2023. A survey on Sign Language machine translation. Expert Systems with Applications 213 (2023), 118993

2023
[22]

Eng-Jon Ong, Oscar Koller, Nicolas Pugeault, and Richard Bowden. 2014. Sign spotting using hierarchical sequential patterns with temporal intervals. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition . 1923– 1930

2014
[23]

Sense. [n. d.]. Sign Language. https://www.sense.org.uk/information-and-advice/ communication/sign-language/. Accessed: 22.03.2024

2024
[24]

Bowen Shi, Diane Brentari, Greg Shakhnarovich, and Karen Livescu. 2022. Open- domain sign language translation learned from online video. arXiv preprint arXiv:2205.12870 (2022)

2022 arXiv
[26]

Garrett Tanzer and Biao Zhang. 2024. YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpus. arXiv preprint arXiv:2407.11144 (2024)

2024 arXiv
[27]

Rachel Tatman. 2017. English Word Frequency. https://www.kaggle.com/datasets/ rtatman/english-word-frequency. Accessed: 30-6-2024

2017
[28]

Gemma Team. 2024. Gemma. doi:10.34740/KAGGLE/M/3301

2024 doi
[29]

The Word Finder. 2024. Random Sentence Generator. https://www.thewordfinder. com/random-sentence-generator/. Accessed: 1-8-2024

2024
[30]

Gül Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, and Andrew Zisserman. 2022. Scaling up sign spotting through sign language dictionaries. International Journal of Computer Vision 130, 6 (2022), 1416–1439

2022
[31]

Ville Viitaniemi, Tommi Jantunen, Leena Savolainen, Matti Karppa, and Jorma Laaksonen. 2014. S-pot–a benchmark in spotting signs within continuous signing. In LREC proceedings. European Language Resources Association (LREC)

2014
[32]

Ryan Wong, Necati Cihan Camgöz, and Richard Bowden. 2022. Hierarchical i3d for sign spotting. InEuropean Conference on Computer Vision Workshops. Springer, 243–255

2022
[33]

Ryan Wong, Necati Cihan Camgöz, and Richard Bowden. 2024. Sign2GPT: Lever- aging Large Language Models for Gloss-Free Sign Language Translation. In The Twelfth International Conference on Learning Representations (ICLR)

2024
[34]

Hee-Deok Yang, Stan Sclaroff, and Seong-Whan Lee. 2008. Sign language spotting with a threshold model based on conditional random fields. IEEE transactions on pattern analysis and machine intelligence 31, 7 (2008), 1264–1277

2008
[35]

Biao Zhang, Mathias Müller, and Rico Sennrich. 2023. SLTUNET: A simple unified model for sign language translation. arXiv preprint arXiv:2305.01778 (2023)

2023 arXiv
[36]

Benjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. 2023. Gloss-free sign language translation: Improving from visual-language pretraining. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 20871–20881

2023
[37]

Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. 2021. Improv- ing sign language translation with monolingual data by sign back-translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 1316–1325

2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [25]

Ozge Mercanoglu Sincan, Necati Cihan Camgoz, and Richard Bowden. 2024. Using an LLM to turn sign spottings into spoken language sentences. arXiv preprint arXiv:2403.10434 (2024)

arXiv 2024

[2] [1]

Samuel Albanie, Gül Varol, Liliane Momeni, Hannah Bull, Triantafyllos Afouras, Himel Chowdhury, Neil Fox, Bencie Woll, Rob Cooper, and Andrew Zisserman McParland. 2021. BBC-Oxford British Sign Language Dataset. arXiv preprint arXiv:2111.03635 (2021)

arXiv 2021

[3] [2]

Sally Beatty. 2024. Tiny but Mighty: The Phi-3 Small Language Models with Big Potential. https://news.microsoft.com/source/features/ai/the-phi-3-small- language-models-with-big-potential/. Accessed: 6-6-2025

work page 2024

[4] [3]

Berndt and James Clifford

Donald J. Berndt and James Clifford. 1994. Using Dynamic Time Warping to Find Patterns in Time Series. In Proceedings of the 3rd international conference on knowledge discovery and data mining

work page 1994

[5] [4]

Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. 2018. Neural Sign Language Translation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 7784–7793

work page 2018

[6] [5]

Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 6299–6308

2017

[7] [6]

Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, and Stephen Lin. 2022. A simple multi-modality transfer learning baseline for sign language translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5120–5130

2022

[8] [7]

Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, and Brian Mak. 2022. Two-stream network for sign language recognition and translation. Advances in Neural Information Processing Systems 35 (2022), 17043–17056

2022