Search Results for author:
Found 78 papers, 24 papers with code
Modelling the Distribution of Human Motion for Sign Language Assessment
Sign Language Assessment (SLA) tools are useful to aid in language learning and are underdeveloped.
Diversity-Aware Sign Language Production through a Pose Encoding Variational Autoencoder
The generator framework is presented as a UNet architecture to ensure spatial preservation of the input pose, and we include the visual features from the variational inference to maintain control over appearance and style.
Sign Stitching: A Novel Approach to Sign Language Production
Then by applying filtering in the frequency domain and resampling each sign we create cohesive natural sequences, that mimic the prosody found in the original data.
Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation
Automatic Sign Language Translation requires the integration of both computer vision and natural language processing to effectively bridge the communication gap between sign and spoken languages.
Gloss-free Sign Language Translation
Sign Language Translation
+1
The Third Monocular Depth Estimation Challenge
This paper discusses the results of the third edition of the Monocular Depth Estimation Challenge (MDEC).
Select and Reorder: A Novel Approach for Neural Sign Language Production
Sign languages, often categorised as low-resource languages, face significant challenges in achieving accurate translation due to the scarcity of parallel annotated datasets.
A Data-Driven Representation for Sign Language Production
By applying Vector Quantisation (VQ) to sign language data, we first learn a codebook of short motions that can be combined to create a natural sequence of sign.
Two Hands Are Better Than One: Resolving Hand to Hand Intersections via Occupancy Networks
This work addresses the intersection of hands by exploiting an occupancy network that represents the hand's volume as a continuous manifold.
Using an LLM to Turn Sign Spottings into Spoken Language Sentences
Sign Language Translation (SLT) is a challenging task that aims to generate spoken language sentences from sign language videos.
Giving a Hand to Diffusion Models: a Two-Stage Approach to Improving Conditional Human Image Generation
Our approach not only enhances the quality of the generated hands but also offers improved control over hand pose, advancing the capabilities of pose-conditioned human image generation.
Kick Back & Relax++: Scaling Beyond Ground-Truth Depth with SlowTV & CribsTV
Self-supervised learning is the key to unlocking generic computer vision systems.
Learnt Contrastive Concept Embeddings for Sign Recognition
In natural language processing (NLP) of spoken languages, word embeddings have been shown to be a useful method to encode the meaning of words.
Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse
Sign Language Translation (SLT) is a challenging task that aims to generate spoken language sentences from sign language videos, both of which have different grammar and word/gloss order.
Denoising Diffusion for 3D Hand Pose Estimation from Images
Hand pose estimation from a single image has many applications.
Improving 3D Pose Estimation for Sign Language
Given a 2D detection of keypoints in the image, we lift the skeleton to 3D using neural networks to predict both the joint rotations and bone lengths.
Gloss Alignment Using Word Embeddings
As a result, research has turned to TV broadcast content as a source of large-scale training data, consisting of both the sign language interpreter and the associated audio subtitle.
Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTV
Unfortunately, existing approaches limit themselves to the automotive domain, resulting in models incapable of generalizing to complex environments such as natural or indoor settings.
Learning Adaptive Neighborhoods for Graph Neural Networks
Our module can be readily integrated into existing pipelines involving graph convolution operations, replacing the predetermined or existing adjacency matrix with one that is learned, and optimized, as part of the general objective.
The Second Monocular Depth Estimation Challenge
This paper discusses the results for the second edition of the Monocular Depth Estimation Challenge (MDEC).
Decision Making for Autonomous Driving in Interactive Merge Scenarios via Learning-based Prediction
This paper presents a decision making approach for autonomous driving, focusing on the complex task of merging into moving traffic where uncertainty emanates from the behavior of other drivers and imperfect sensor measurements.
Novel View Synthesis of Humans using Differentiable Rendering
We show how our approach can be used for motion transfer between individuals; novel view synthesis of individuals captured from just a single camera; to synthesize individuals from any virtual viewpoint; and to re-render people in novel poses.
The Monocular Depth Estimation Challenge
This challenge evaluated the progress of self-supervised monocular depth estimation on the challenging SYNS-Patches dataset.
Hierarchical I3D for Sign Spotting
Most of the vision-based sign language research to date has focused on Isolated Sign Language Recognition (ISLR), where the objective is to predict a single sign class given a short video clip.
Changing the Representation: Examining Language Representation for Neural Sign Language Production
We use language models such as BERT and Word2Vec to create better sentence level embeddings, and apply several tokenization techniques, demonstrating how these improve performance on the low resource translation task of Text to Gloss.
Deconstructing Self-Supervised Monocular Reconstruction: The Design Decisions that Matter
It is likely that many papers were not only optimized for particular datasets, but also for errors in the data and evaluation criteria.
AFT-VO: Asynchronous Fusion Transformers for Multi-View Visual Odometry Estimation
To address this limitation, in this work, we propose AFT-VO, a novel transformer-based sensor fusion architecture to estimate VO from multiple sensors.
Medusa: Universal Feature Learning via Attentional Multitasking
We argue that MTL is a stepping stone towards universal feature learning (UFL), which is the ability to learn generic features that can be applied to new tasks without retraining.
"The Pedestrian next to the Lamppost" Adaptive Object Graphs for Better Instantaneous Mapping
Estimating a semantically segmented bird's-eye-view (BEV) map from a single image has become a popular technique for autonomous control and navigation.
Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production
To learn sign co-articulation, we propose a novel Frame Selection Network (FS-Net) that improves the temporal alignment of interpolated dictionary signs to continuous signing sequences.
A Free Lunch with Influence Functions? Improving Neural Network Estimates with Concepts from Semiparametric Statistics
Unfortunately, they are unlikely to be sufficiently flexible to be able to adequately model real-world phenomena, and may yield biased estimates.
MDN-VO: Estimating Visual Odometry with Confidence
Visual Odometry (VO) is used in many applications including robotics and autonomous systems.
Multi-Camera Sensor Fusion for Visual Odometry using Deep Uncertainty Estimation
To address this issue, we propose a deep sensor fusion framework which estimates vehicle motion using both pose and uncertainty estimations from multiple on-board cameras.
Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into Sign Language Production
Recent approaches to Sign Language Production (SLP) have adopted spoken language Neural Machine Translation (NMT) architectures, applied without sign-specific modifications.
Human Pose Manipulation and Novel View Synthesis using Differentiable Rendering
We show how our approach can be used for motion transfer between individuals; novel view synthesis of individuals captured from just a single camera; to synthesize individuals from any virtual viewpoint; and to re-render people in novel poses.
Translating Images into Maps
We show how a novel form of transformer network can be used to map from images and video directly to an overhead map or bird's-eye-view (BEV) of the world, in a single end-to-end network.
Improving Robot Localisation by Ignoring Visual Distraction
Attention is an important component of modern deep learning.
Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives
Using a progressive transformer for the translation sub-task, we propose a novel Mixture of Motion Primitives (MoMP) architecture for sign language animation.
AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisation
To tackle SLVA, we propose AnonySign, a novel automatic approach for visual anonymisation of sign language data.
Looking for the Signs: Identifying Isolated Sign Instances in Continuous Video Footage
In this paper, we focus on the task of one-shot sign spotting, i. e. given an example of an isolated sign (query), we want to identify whether/where this sign appears in a continuous, co-articulated sign language video (target).
Adversarial Mixture Density Networks: Learning to Drive Safely from Collision Data
By penalising the safe action distribution based on its similarity to the unsafe action distribution when training on the collision dataset, a more robust and safe control policy is obtained.
ARC: Adversarially Robust Control Policies for Autonomous Vehicles
By training the protagonist against an ensemble of adversaries, it learns a significantly more robust control policy, which generalises to a variety of adversarial strategies.
Markov Localisation using Heatmap Regression and Deep Convolutional Odometry
Recent advances in deep learning hardware allow large likelihood volumes to be stored directly on the GPU, along with the hardware necessary to efficiently perform GPU-bound 3D convolutions and this obviates many of the disadvantages of grid based methods.
Content4All Open Research Sign Language Translation Datasets
Computational sign language research lacks the large-scale datasets that enables the creation of useful reallife applications.
Skeletor: Skeletal Transformers for Robust Body-Pose Estimation
Skeletor can achieve this as it implicitly learns the spatio-temporal context of human motion via a transformer based neural network.
Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition
Basically, skeletal representations generalize over an individual's appearance and background, allowing us to focus on the recognition of motion.
Shadow-Mapping for Unsupervised Neural Causal Discovery
An important goal across most scientific fields is the discovery of causal structures underling a set of observations.
There and Back Again: Self-supervised Multispectral Correspondence Estimation
In this work, we aim to address the dense correspondence estimation problem in a way that generalizes to more than one spectrum.
Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages
We compare models with and without safety cages, as well as models with optimal and constrained model parameters, and show that the weak supervision consistently improves the safety of exploration, speed of convergence, and model performance.
VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Experts
Given that supervision is often expensive or infeasible to acquire, we choose to incorporate structural inductive bias and present an unsupervised, deep State-Space-Model for Video Disentanglement (VDSM).
Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks
Sign languages are multi-channel visual languages, where signers use a continuous 3D space to communicate. Sign Language Production (SLP), the automatic translation from spoken to sign languages, must embody both the continuous articulation and full morphology of sign to be truly understandable by the Deaf community.
D'ya like DAGs? A Survey on Structure Learning and Causal Discovery
Causal reasoning is a crucial part of science and human intelligence.
Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video
To be truly understandable and accepted by Deaf communities, an automatic Sign Language Production (SLP) system must generate a photo-realistic signer.
Targeted VAE: Variational and Targeted Learning for Causal Inference
Undertaking causal inference with observational data is incredibly useful across a wide range of tasks including the development of medical treatments, advertisements and marketing, and policy making.
Targeted VAE: Structured Inference and Targeted Learning for Causal Parameter Estimation
Undertaking causal inference with observational data is extremely useful across a wide range of domains including the development of medical treatments, advertisements and marketing, and policy making.
Multi-channel Transformers for Multi-articulatory Sign Language Translation
Sign languages use multiple asynchronous information channels (articulators), not just the hands but also the face and body, which computational approaches often ignore.
Adversarial Training for Multi-Channel Sign Language Production
Sign Languages are rich multi-channel languages, requiring articulation of both manual (hands) and non-manual (face and body) features in a precise, intricate manner.
Progressive Transformers for End-to-End Sign Language Production
The goal of automatic Sign Language Production (SLP) is to translate spoken language to a continuous stream of sign language video at a level comparable to a human translator.
Same Features, Different Day: Weakly Supervised Feature Learning for Seasonal Invariance
The aim of this paper is to provide a dense feature representation that can be used to perform localization, sparse matching or image retrieval, regardless of the current seasonal or temporal appearance.
DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learning
In the current monocular depth research, the dominant approach is to employ unsupervised training on large datasets, driven by warped photometric consistency.
Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation
We report state-of-the-art sign language recognition and translation results achieved by our Sign Language Transformers.
Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies
As the networks used to obtain state-of-the-art results become increasingly deep and complex, the rules they have learned and how they operate become more challenging to understand.
NestedVAE: Isolating Common Factors via Weak Supervision
Two outer VAEs with shared weights attempt to reconstruct the input and infer a latent space, whilst a nested VAE attempts to reconstruct the latent representation of one image, from the latent representation of its paired image.
A Survey of Deep Learning Applications to Autonomous Vehicle Control
However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalising previously learned rules to new scenarios.
Gated Variational AutoEncoders: Incorporating Weak Supervision to Encourage Disentanglement
However, there is some debate about how to encourage disentanglement with VAEs and evidence indicates that existing implementations of VAEs do not achieve disentanglement consistently.
Weakly-Supervised 3D Pose Estimation from a Single Image using Multi-View Consistency
We present a novel data-driven regularizer for weakly-supervised learning of 3D human pose estimation that eliminates the drift problem that affects existing approaches.
Scale-Adaptive Neural Dense Features: Learning via Hierarchical Context Aggregation
In all cases, we show how incorporating SAND features results in better or comparable results to the baseline, whilst requiring little to no additional training.
Localisation via Deep Imagination: learn the features not the map
In order to build the embedded map, we train a deep Siamese Fully Convolutional U-Net to perform dense feature extraction.
Neural Sign Language Translation
SLR seeks to recognize a sequence of continuous signs but neglects the underlying rich grammatical and linguistic structures of sign language that differ from spoken language.
Ranked #11 on
Sign Language Translation
on RWTH-PHOENIX-Weather 2014 T
Taking the Scenic Route to 3D: Optimising Reconstruction From Moving Cameras
This approach is ill-suited for reconstruction applications, where learning about the environment is more valuable than speed of traversal.
SubUNets: End-To-End Hand Shape and Continuous Sign Language Recognition
We propose a novel deep learning approach to solve simultaneous alignment and recognition problems (referred to as "Sequence-to-sequence" learning).
Ranked #20 on
Sign Language Recognition
on RWTH-PHOENIX-Weather 2014
SeDAR - Semantic Detection and Ranging: Humans can localise without LiDAR, can robots?
Similarly, we do not extrude the 2D geometry present in the floorplan into 3D and try to align it to the real-world.
Image and Video Mining through Online Learning
On the UCF11 video dataset, the accuracy is 86. 7% despite using only 90 labelled examples from a dataset of over 1200 videos, instead of the standard 1122 training videos.
Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data Is Continuous and Weakly Labelled
Furthermore, we demonstrate its use in continuous sign language recognition on two publicly available large sign language data sets, where it outperforms the current state-of-the-art by a large margin.
Exploring Causal Relationships in Visual Object Tracking
We show that the location predictions are robust to camera shake and sud- den motion, which is invaluable for any tracking algorithm and demonstrate this by applying causal prediction to two state-of-the-art trackers.
Exploiting High Level Scene Cues in Stereo Reconstruction
We present a novel approach to 3D reconstruction which is inspired by the human visual system.
Sign Spotting using Hierarchical Sequential Patterns with Temporal Intervals
This paper tackles the problem of spotting a set of signs occuring in videos with sequences of signs.
Hollywood 3D: Recognizing Actions in 3D Natural Scenes
In addition, two state of the art action recognition algorithms are extended to make use of the 3D data, and five new interest point detection strategies are also proposed, that extend to the 3D data.
