REVIEW 1 cited by
Testing MediaPipe Holistic for Linguistic Analysis of Nonmanual Markers in Sign Languages
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Advances in Deep Learning have made possible reliable landmark tracking of human bodies and faces that can be used for a variety of tasks. We test a recent Computer Vision solution, MediaPipe Holistic (MPH), to find out if its tracking of the facial features is reliable enough for a linguistic analysis of data from sign languages, and compare it to an older solution (OpenFace, OF). We use an existing data set of sentences in Kazakh-Russian Sign Language and a newly created small data set of videos with head tilts and eyebrow movements. We find that MPH does not perform well enough for linguistic analysis of eyebrow movement - but in a different way from OF, which is also performing poorly without correction. We reiterate a previous proposal to train additional correction models to overcome these limitations.
Forward citations
Cited by 1 Pith paper
-
SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction
A masked cluster-prediction transformer over four sign-language streams sets state-of-the-art results on multiple ASL translation and recognition benchmarks using only public pre-training data.
Discussion (0). Continue with ORCID to comment.