Sign Language Translation
34 papers with code • 5 benchmarks • 13 datasets
Given a video containing sign language, the task is to predict the translation into (written) spoken language.
Image credit: How2Sign
Benchmarks
These leaderboards are used to track progress in Sign Language Translation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Datasets
WLASL
How2Sign
OpenASL
BosphorusSign22k
Content4All
LSFB Datasets
ASLG-PC12
Latest papers
CorrNet+: Sign Language Recognition and Translation via Spatial-Temporal Correlation
In specific, CorrNet+ employs a correlation module and an identification module to build human body trajectories.
A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars
FangyunWei/SLRT
•
•
The objective of this paper is to develop a functional system for translating spoken languages into sign languages, referred to as Spoken2Sign translation.
Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment
The first KL divergence optimizes the conditional variational autoencoder and regularizes the encoder outputs, while the second KL divergence performs a self-distillation from the posterior path to the prior path, ensuring the consistency of decoder outputs.
Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Models
pooyafayyaz/fingerspelling-posenet
•
•
We also propose a novel two-stage inference approach that re-ranks the hypotheses using the language model capabilities of the decoder.
JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing
Advancements in sign language processing have been hindered by a lack of sufficient data, impeding progress in recognition, translation, and production tasks.
SignBank+: Preparing a Multilingual Sign Language Dataset for Machine Translation Using Large Language Models
We introduce SignBank+, a clean version of the SignBank dataset, optimized for machine translation between spoken language text and SignWriting, a phonetic sign language writing system.
Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining
zhoubenjia/gfslt-vlp
•
•
ICCV 2023
Many previous methods employ an intermediate representation, i. e., gloss sequences, to facilitate SLT, thus transforming it into a two-stage task of sign language recognition (SLR) followed by sign language translation (SLT).
Gloss Attention for Gloss-free Sign Language Translation
yinaoxiong/gaslt
•
•
CVPR 2023
We find that it can provide two aspects of information for the model, 1) it can help the model implicitly learn the location of semantic boundaries in continuous sign language videos, 2) it can help the model understand the sign language video globally.
ISLTranslate: Dataset for Translating Indian Sign Language
To the best of our knowledge, it is the largest translation dataset for continuous Indian Sign Language.
An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation
Sign language translation systems are complex and require many components.
