Search Results for author:
Found 20 papers, 7 papers with code
Open-Domain Sign Language Translation Learned from Online Video
Existing work on sign language translation--that is, translation from sign language videos into sentences in a written language--has focused mainly on (1) data collected in a controlled environment or (2) data in a specific domain, which limits the applicability to real-world settings.
Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT
This paper investigates self-supervised pre-training for audio-visual speaker representation learning where a visual stream showing the speaker's mouth area is used alongside speech as inputs.
Searching for fingerspelled content in American Sign Language
This is an important task since significant content in sign language is often conveyed via fingerspelling, and to our knowledge the task has not been studied before.
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction
The lip-reading WER is further reduced to 26. 9% when using all 433 hours of labeled data from LRS3 and combined with self-training.
Ranked #1 on
Lipreading
on LRS3-TED
(using extra training data)
Robust Self-Supervised Audio-Visual Speech Recognition
Audio-based automatic speech recognition (ASR) degrades significantly in noisy environments and is particularly vulnerable to interfering speech, as the model cannot determine which speaker to transcribe.
Ranked #1 on
Audio-Visual Speech Recognition
on LRS3-TED
Audio-Visual Speech Recognition
Automatic Speech Recognition
+3
Hierarchical Graph Networks for 3D Human Pose Estimation
Recent 2D-to-3D human pose estimation works tend to utilize the graph structure formed by the topology of the human skeleton.
Multi-dataset Pretraining: A Unified Model for Semantic Segmentation
This is achieved by first pretraining the network via the proposed pixel-to-prototype contrastive loss over multiple datasets regardless of their taxonomy labels, and followed by fine-tuning the pretrained model over specific dataset as usual.
Fingerspelling Detection in American Sign Language
We propose a benchmark and a suite of evaluation metrics, some of which reflect the effect of detection on the downstream fingerspelling recognition task.
Domain wall topological entanglement entropy
We derive a universal correction to the ground-state entanglement entropy, which is equal to the logarithm of the total quantum dimension of a set of superselection sectors localized on the domain wall.
Strongly Correlated Electrons High Energy Physics - Theory Quantum Physics
Whole-Word Segmental Speech Recognition with Acoustic Word Embeddings
We find that word error rate can be reduced by a large margin by pre-training the acoustic segment representation with AWEs, and additional (smaller) gains can be obtained by pre-training the word prediction layer with AGWEs.
A Cross-Task Analysis of Text Span Representations
Many natural language processing (NLP) tasks involve reasoning with textual spans, including question answering, entity recognition, and coreference resolution.
Few-shot acoustic event detection via meta-learning
We study few-shot acoustic event detection (AED) in this paper.
Latency-Aware Differentiable Neural Architecture Search
However, these methods suffer the difficulty in optimizing network, so that the searched network is often unfriendly to hardware.
Fingerspelling recognition in the wild with iterative visual attention
In this paper we focus on recognition of fingerspelling sequences in American Sign Language (ASL) videos collected in the wild, mainly from YouTube and Deaf social media.
Compression of Acoustic Event Detection Models With Quantized Distillation
Acoustic Event Detection (AED), aiming at detecting categories of events based on audio signals, has found application in many intelligent systems.
Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training
In this paper, we present a compression approach based on the combination of low-rank matrix factorization and quantization training, to reduce complexity for neural network based acoustic event detection (AED) models.
Semi-supervised Acoustic Event Detection based on tri-training
This paper presents our work of training acoustic event detection (AED) models using unlabeled dataset.
On the Contributions of Visual and Textual Supervision in Low-Resource Semantic Speech Retrieval
Recent work has shown that speech paired with images can be used to learn semantically meaningful speech representations even without any textual supervision.
American Sign Language fingerspelling recognition in the wild
As the first attempt at fingerspelling recognition in the wild, this work is intended to serve as a baseline for future work on sign language recognition in realistic conditions.
Multitask training with unlabeled data for end-to-end sign language fingerspelling recognition
We introduce a model for fingerspelling recognition that addresses these issues.
