Search Results for author:
Found 22 papers, 8 papers with code
Scaling up sign spotting through sign language dictionaries
The focus of this work is $\textit{sign spotting}$ - given a video of an isolated sign, our task is to identify $\textit{whether}$ and $\textit{where}$ it has been signed in a continuous, co-articulated sign language video.
Audio-Visual Synchronisation in the wild
Finally, we set the first benchmark for general audio-visual synchronisation with over 160 diverse classes in the new VGG-Sound Sync video dataset.
BBC-Oxford British Sign Language Dataset
In this work, we introduce the BBC-Oxford British Sign Language (BOBSL) dataset, a large-scale video collection of British Sign Language (BSL).
Visual Keyword Spotting with Attention
In this paper, we consider the task of spotting spoken keywords in silent video sequences -- also known as visual keyword spotting.
Ranked #1 on
Visual Keyword Spotting
on LRS2
Sub-word Level Lip Reading With Visual Attention
To this end, we make the following contributions: (1) we propose an attention-based pooling mechanism to aggregate visual speech representations; (2) we use sub-word units for lip reading for the first time and show that this allows us to better model the ambiguities of the task; (3) we propose a model for Visual Speech Detection (VSD), trained on top of the lip reading network.
Ranked #1 on
Lipreading
on LRS2
(using extra training data)
Audio-Visual Active Speaker Detection
Automatic Speech Recognition
+3
Aligning Subtitles in Sign Language Videos
The goal of this work is to temporally align asynchronous subtitles in sign language videos.
Self-supervised object detection from audio-visual correspondence
We tackle the problem of learning object detectors without supervision.
Localizing Visual Sounds the Hard Way
We show that our algorithm achieves state-of-the-art performance on the popular Flickr SoundNet dataset.
Read and Attend: Temporal Localisation in Sign Language Videos
Our contributions are as follows: (1) we demonstrate the ability to leverage large quantities of continuous signing videos with weakly-aligned subtitles to localise signs in continuous sign language; (2) we employ the learned attention to automatically generate hundreds of thousands of annotations for a large sign vocabulary; (3) we collect a set of 37K manually verified sign instances across a vocabulary of 950 sign classes to support our study of sign language recognition; (4) by training on the newly annotated data from our method, we outperform the prior state of the art on the BSL-1K sign language recognition benchmark.
Watch, read and lookup: learning to spot signs from multiple supervisors
The focus of this work is sign spotting - given a video of an isolated sign, our task is to identify whether and where it has been signed in a continuous, co-articulated sign language video.
Seeing wake words: Audio-visual Keyword Spotting
The goal of this work is to automatically determine whether and when a word of interest is spoken by a talking face, with or without the audio.
Self-Supervised Learning of Audio-Visual Objects from Video
Our objective is to transform a video into a set of discrete audio-visual objects using self-supervised learning.
BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues
Recent progress in fine-grained gesture and action classification, and machine translation, point to the possibility of automated sign language recognition becoming a reality.
Ranked #2 on
Sign Language Recognition
on WLASL-2000
Spot the conversation: speaker diarisation in the wild
Finally, we use this pipeline to create a large-scale diarisation dataset called VoxConverse, collected from 'in the wild' videos, which we will release publicly to the research community.
ASR is all you need: cross-modal distillation for lip reading
The goal of this work is to train strong models for visual speech recognition without requiring human annotated ground truth data.
Ranked #10 on
Lipreading
on LRS2
(using extra training data)
My lips are concealed: Audio-visual speech enhancement through obstructions
To this end we introduce a deep audio-visual speech enhancement network that is able to separate a speaker's voice by conditioning on both the speaker's lip movements and/or a representation of their voice.
Deep Audio-Visual Speech Recognition
The goal of this work is to recognise phrases and sentences being spoken by a talking face, with or without the audio.
Ranked #5 on
Audio-Visual Speech Recognition
on LRS3-TED
LRS3-TED: a large-scale dataset for visual speech recognition
This paper introduces a new multi-modal dataset for visual and audio-visual speech recognition.
Audio-Visual Speech Recognition
Visual Speech Recognition
+1
Deep Lip Reading: a comparison of models and an online application
The goal of this paper is to develop state-of-the-art models for lip reading -- visual speech recognition.
The Conversation: Deep Audio-Visual Speech Enhancement
Our goal is to isolate individual speakers from multi-talker simultaneous speech in videos.
Counterfactual Multi-Agent Policy Gradients
COMA uses a centralised critic to estimate the Q-function and decentralised actors to optimise the agents' policies.
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
Many real-world problems, such as network packet routing and urban traffic control, are naturally modeled as multi-agent reinforcement learning (RL) problems.
