Pith. sign in

REVIEW 1 cited by

FSboard: Over 3 million characters of ASL fingerspelling collected via smartphones

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.15806 v1 pith:EAX7IJQG submitted 2024-07-22 cs.CV cs.CL

Manfred Georg , Garrett Tanzer , Saad Hassan , Maximus Shengelia , Esha Uboweja , Sam Sepah , Sean Forbes , Thad Starner This is my paper · ORCID
classification cs.CVcs.CL
keywords fingerspellingfsboardsignsignerscharacterscollecteddatasetdeaf
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Progress in machine understanding of sign languages has been slow and hampered by limited data. In this paper, we present FSboard, an American Sign Language fingerspelling dataset situated in a mobile text entry use case, collected from 147 paid and consenting Deaf signers using Pixel 4A selfie cameras in a variety of environments. Fingerspelling recognition is an incomplete solution that is only one small part of sign language translation, but it could provide some immediate benefit to Deaf/Hard of Hearing signers as more broadly capable technology develops. At >3 million characters in length and >250 hours in duration, FSboard is the largest fingerspelling recognition dataset to date by a factor of >10x. As a simple baseline, we finetune 30 Hz MediaPipe Holistic landmark inputs into ByT5-Small and achieve 11.1% Character Error Rate (CER) on a test set with unique phrases and signers. This quality degrades gracefully when decreasing frame rate and excluding face/body landmarks: plausible optimizations to help models run on device in real time.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction

    cs.CL 2024-11 conditional novelty 6.0 of 10

    A masked cluster-prediction transformer over four sign-language streams sets state-of-the-art results on multiple ASL translation and recognition benchmarks using only public pre-training data.

Pith tools