Search Results for author:
Found 26 papers, 12 papers with code
Real-Time Multilingual Sign Language Processing
We contribute foundational libraries and resources to the SLP community, thereby setting the stage for a more in-depth exploration of the tasks of sign language translation and production.
signwriting-evaluation: Effective Sign Language Evaluation via SignWriting
The lack of automatic evaluation metrics tailored for SignWriting presents a significant obstacle in developing effective transcription and translation models for signed languages.
Pose-Based Sign Language Appearance Transfer
We introduce a method for transferring the signer's appearance in sign language skeletal poses while preserving the sign content.
SignCLIP: Connecting Text and Sign Language by Contrastive Learning
We present SignCLIP, which re-purposes CLIP (Contrastive Language-Image Pretraining) to project spoken language text and sign language videos, two classes of natural languages of distinct modalities, into the same space.
Optimizing Hand Region Detection in MediaPipe Holistic Full-Body Pose Estimation to Improve Accuracy and Avoid Downstream Errors
This paper addresses a critical flaw in MediaPipe Holistic's hand Region of Interest (ROI) prediction, which struggles with non-ideal hand orientations, affecting sign language recognition accuracy.
Linguistically Motivated Sign Language Segmentation
We find that introducing BIO tagging is necessary to model sign boundaries.
pose-format: Library for Viewing, Augmenting, and Handling .pose Files
The library includes a specialized file format that encapsulates various types of pose data, accommodating multiple individuals and an indefinite number of time frames, thus proving its utility for both image and video data.
sign.mt: Real-Time Multilingual Sign Language Translation Application
Harnessing state-of-the-art open-source models, this tool aims to address the communication divide between the hearing and the deaf, facilitating seamless translation in both spoken-to-signed and signed-to-spoken translation directions.
SignBank+: Preparing a Multilingual Sign Language Dataset for Machine Translation Using Large Language Models
We introduce SignBank+, a clean version of the SignBank dataset, optimized for machine translation between spoken language text and SignWriting, a phonetic sign language writing system.
Addressing the Blind Spots in Spoken Language Processing
This paper explores the critical but often overlooked role of non-verbal cues, including co-speech gestures and facial expressions, in human communication and their implications for Natural Language Processing (NLP).
An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation
Sign language translation systems are complex and require many components.
At Your Fingertips: Extracting Piano Fingering Instructions from Videos
Piano fingering -- knowing which finger to use to play each note in a musical piece, is a hard and important skill to master when learning to play the piano.
Considerations for meaningful sign language machine translation based on glosses
Automatic sign language processing is gaining popularity in Natural Language Processing (NLP) research (Yin et al., 2021).
Ham2Pose: Animating Sign Language Notation into Pose Sequences
Additionally, we offer a new distance measurement that considers missing keypoints, to measure the distance between pose sequences using DTW-MJE.
Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting
This paper presents work on novel machine translation (MT) systems between spoken and signed languages, where signed languages are represented in SignWriting, a sign language writing system.
cofga: A Dataset for Fine Grained Classification of Objects from Aerial Imagery
Detection and classification of objects in overhead images are two important and challenging problems in computer vision.
Data Augmentation for Sign Language Gloss Translation
Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss-to-text translation, where a gloss is a sequence of transcribed spoken-language words in the order in which they are signed.
Data Augmentation
Low Resource Neural Machine Translation
+5
Including Signed Languages in Natural Language Processing
Signed languages are the primary means of communication for many deaf and hard of hearing individuals.
Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition
Basically, skeletal representations generalize over an individual's appearance and background, allowing us to focus on the recognition of motion.
Real-Time Sign Language Detection using Human Pose Estimation
We propose a lightweight real-time sign language detection model, as we identify the need for such a case in videoconferencing.
At Your Fingertips: Automatic Piano Fingering Detection
Automatic Piano Fingering is a hard task which computers can learn using data.
Improving Quality and Efficiency in Plan-based Neural Data-to-Text Generation
We follow the step-by-step approach to neural data-to-text generation we proposed in Moryossef et al (2019), in which the generation process is divided into a text-planning stage followed by a plan-realization stage.
Filling Gender \& Number Gaps in Neural Machine Translation with Black-box Context Injection
When translating from a language that does not morphologically mark information such as gender and number into a language that does, translation systems must {``}guess{''} this missing information, often leading to incorrect translations in the given context.
Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation
We propose to split the generation process into a symbolic text-planning stage that is faithful to the input, followed by a neural generation stage that focuses only on realization.
Ranked #15 on
Data-to-Text Generation
on WebNLG
Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection
When translating from a language that does not morphologically mark information such as gender and number into a language that does, translation systems must "guess" this missing information, often leading to incorrect translations in the given context.
ABI Neural Ensemble Model for Gender Prediction Adapt Bar-Ilan Submission for the CLIN29 Shared Task on Gender Prediction
In contradiction with the results described in previous comparable shared tasks, our neural models performed better than our best traditional approaches with our best feature set-up.
