Sign Language Translation
35 papers with code • 5 benchmarks • 13 datasets
Given a video containing sign language, the task is to predict the translation into (written) spoken language.
Image credit: How2Sign
Benchmarks
These leaderboards are used to track progress in Sign Language Translation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Datasets
WLASL
How2Sign
OpenASL
BosphorusSign22k
Content4All
LSFB Datasets
ASLG-PC12
Latest papers with no code
Toward American Sign Language Processing in the Real World: Data, Tasks, and Methods
To address the problem of searching for fingerspelled keywords in raw sign language videos, we propose a novel method that jointly localizes and matches fingerspelling segments to text.
Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse
Sign Language Translation (SLT) is a challenging task that aims to generate spoken language sentences from sign language videos, both of which have different grammar and word/gloss order.
A two-way translation system of Chinese sign language based on computer vision
As the main means of communication for deaf people, sign language has a special grammatical order, so it is meaningful and valuable to develop a real-time translation system for sign language.
SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding
In our framework, the hand pose is regarded as a visual token, which is derived from an off-the-shelf detector.
A Comparative Analysis of Techniques and Algorithms for Recognising Sign Language
Sign language is a visual language that enhances communication between people and is frequently used as the primary form of communication by people with hearing loss.
SignNet: Single Channel Sign Generation using Metric Embedded Learning
In the task of gloss to pose, SignNet performed as well as its state-of-the-art (SoTA) counterparts and outperformed them in the task of text to pose, by showing noteworthy enhancements in BLEU 1 - BLEU 4 scores (BLEU 1: 31->39; ~26% improvement and BLEU 4: 10. 43->11. 84; ~14\% improvement) when tested on the popular RWTH PHOENIX-Weather-2014T benchmark dataset
Considerations for meaningful sign language machine translation based on glosses
Automatic sign language processing is gaining popularity in Natural Language Processing (NLP) research (Yin et al., 2021).
Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation
Therefore, we propose to introduce additional word-level semantic knowledge of sign language linguistics to assist in improving current end-to-end neural SLT models.
Clean Text and Full-Body Transformer: Microsoft's Submission to the WMT22 Shared Task on Sign Language Translation
The BLEU score is further improved to 1. 08 on the dev set by applying features extracted from a lip reading model.
Changing the Representation: Examining Language Representation for Neural Sign Language Production
We use language models such as BERT and Word2Vec to create better sentence level embeddings, and apply several tokenization techniques, demonstrating how these improve performance on the low resource translation task of Text to Gloss.
