Search Results for author:
Found 18 papers, 5 papers with code
Egocentric Auditory Attention Localization in Conversations
In a noisy conversation environment such as a dinner party, people often exhibit selective auditory attention, or the ability to focus on a particular speaker while tuning out others.
Novel-View Acoustic Synthesis
We introduce the novel-view acoustic synthesis (NVAS) task: given the sight and sound observed at a source viewpoint, can we synthesize the sound of that scene from an unseen target viewpoint?
Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations
Can conversational videos captured from multiple egocentric viewpoints reveal the map of a scene in a cost-efficient way?
LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders
Audio-visual speech enhancement aims to extract clean speech from a noisy environment by leveraging not only the audio itself but also the target speaker's lip movements.
Leveraging Heteroscedastic Uncertainty in Learning Complex Spectral Mapping for Single-channel Speech Enhancement
During training, our approach augments a model learning complex spectral mapping with a temporary submodel to predict the covariance of the enhancement error at each time-frequency bin.
Towards Improved Room Impulse Response Estimation for Speech Recognition
We propose a novel approach for blind room impulse response (RIR) estimation systems in the context of a downstream application scenario, far-field automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing
RemixIT is based on a continuous self-training scheme in which a pre-trained teacher model on out-of-domain data infers estimated pseudo-target signals for in-domain mixtures.
Deep Impulse Responses: Estimating and Parameterizing Filters with Deep Networks
Impulse response estimation in high noise and in-the-wild settings, with minimal control of the underlying data distributions, is a challenging problem.
Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localization
We propose a novel end-to-end deep learning approach that is able to give robust voice activity detection and localization results.
Ranked #1 on
Active Speaker Localization
on EasyCom
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Filtered Noise Shaping for Time Domain Room Impulse Response Estimation From Reverberant Speech
Deep learning approaches have emerged that aim to transform an audio signal so that it sounds as if it was recorded in the same room as a reference recording, with applications both in audio post-production and augmented reality.
EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments
In this work, we describe, evaluate and release a dataset that contains over 5 hours of multi-modal data useful for training and testing algorithms for the application of improving conversations for an AR glasses wearer.
Ranked #1 on
Speech Enhancement
on EasyCom
Do sound event representations generalize to other audio tasks? A case study in audio transfer learning
We also provide insights into the attributes of sound event representations that enable such efficient information transfer.
Egocentric Pose Estimation from Human Vision Span
Existing approaches either use a narrow field of view front facing camera that barely captures the wearer, or an extruded head-mounted top-down camera for maximal wearer visibility.
Audio-Visual Floorplan Reconstruction
Given only a few glimpses of an environment, how much can we infer about its entire floorplan?
A Sequential Self Teaching Approach for Improving Generalization in Sound Event Recognition
An important problem in machine auditory perception is to recognize and detect sound events.
Ranked #34 on
Audio Classification
on AudioSet
SoundSpaces: Audio-Visual Navigation in 3D Environments
Moving around in the world is naturally a multisensory experience, but today's embodied agents are deaf---restricted to solely their visual perception of the environment.
Secost: Sequential co-supervision for large scale weakly labeled audio event detection
Weakly supervised learning algorithms are critical for scaling audio event detection to several hundreds of sound categories.
