Search Results for author:
Found 45 papers, 14 papers with code
Neural Acoustic Context Field: Rendering Realistic Room Impulse Response With Neural Fields
Room impulse response (RIR), which measures the sound propagation within an environment, is critical for synthesizing high-fidelity audio for a given environment.
DAVIS: High-Quality Audio-Visual Separation with Generative Diffusion Models
We propose DAVIS, a Diffusion model-based Audio-VIusal Separation framework that solves the audio-visual sound source separation task through a generative manner.
TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudio
To enable this, a variety of metrics to measure quality and intelligibility under different assumptions have been developed.
Egocentric Audio-Visual Object Localization
In this paper, we explore the challenging egocentric audio-visual object localization task and observe that 1) egomotion commonly exists in first-person recordings, even within a short duration; 2) The out-of-view sound components can be created while wearers shift their attention.
TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement
We propose an objective for perceptual quality based on temporal acoustic parameters.
PAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement
We can add this criterion as an auxiliary loss to any model that produces speech, to optimize speech outputs to match the values of clean speech in these features.
AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis
Human perception of the complex world relies on a comprehensive analysis of multi-modal signals, and the co-occurrences of audio and video signals provide humans with rich cues.
Rethinking complex-valued deep neural networks for monaural speech enhancement
By comparing complex- and real-valued versions of fundamental building blocks in the recently developed gated convolutional recurrent network (GCRN), we show how different mechanisms for basic blocks affect the performance.
LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders
Audio-visual speech enhancement aims to extract clean speech from a noisy environment by leveraging not only the audio itself but also the target speaker's lip movements.
Leveraging Heteroscedastic Uncertainty in Learning Complex Spectral Mapping for Single-channel Speech Enhancement
During training, our approach augments a model learning complex spectral mapping with a temporary submodel to predict the covariance of the enhancement error at each time-frequency bin.
Improving Speech Enhancement through Fine-Grained Speech Characteristics
We first identify key acoustic parameters that have been found to correlate well with voice quality (e. g. jitter, shimmer, and spectral flux) and then propose objective functions which are aimed at reducing the difference between clean speech and enhanced speech with respect to these features.
SAQAM: Spatial Audio Quality Assessment Metric
Audio quality assessment is critical for assessing the perceptual realism of sounds.
Speech Quality Assessment through MOS using Non-Matching References
Human judgments obtained through Mean Opinion Scores (MOS) are the most reliable way to assess the quality of speech signals.
Curriculum optimization for low-resource speech recognition
Modern end-to-end speech recognition models show astonishing results in transcribing audio signals into written text.
RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing
RemixIT is based on a continuous self-training scheme in which a pre-trained teacher model on out-of-domain data infers estimated pseudo-target signals for in-domain mixtures.
The impact of removing head movements on audio-visual speech enhancement
This paper investigates the impact of head movements on audio-visual speech enhancement (AVSE).
Continual self-training with bootstrapped remixing for speech enhancement
Specifically, a separation teacher model is pre-trained on an out-of-domain dataset and is used to infer estimated target signals for a batch of in-domain mixtures.
Conformer-Based Self-Supervised Learning for Non-Speech Audio Tasks
While self-supervised speech representation learning has been popular in the speech research community, very few works have comprehensively analyzed audio representation learning for non-speech audio tasks.
Ranked #5 on
Audio Classification
on Balanced Audio Set
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
NORESQA: A Framework for Speech Quality Assessment using Non-Matching References
We show that neural networks trained using our framework produce scores that correlate well with subjective mean opinion scores (MOS) and are also competitive to methods such as DNSMOS, which explicitly relies on MOS from humans for training networks.
Incorporating Real-world Noisy Speech in Neural-network-based Speech Enhancement Systems
Supervised speech enhancement relies on parallel databases of degraded speech signals and their clean reference signals during training.
Online Self-Attentive Gated RNNs for Real-Time Speaker Separation
Our stateful implementation for online separation leads to a minor drop in performance compared to the offline model; 0. 8dB for monaural inputs and 0. 3dB for binaural inputs while reaching a real-time factor of 0. 65.
Do sound event representations generalize to other audio tasks? A case study in audio transfer learning
We also provide insights into the attributes of sound event representations that enable such efficient information transfer.
DPLM: A Deep Perceptual Spatial-Audio Localization Metric
Subjective evaluations are critical for assessing the perceptual realism of sounds in audio-synthesis driven technologies like augmented and virtual reality.
A Low-Delay MAC for IoT Applications: Decentralized Optimal Scheduling of Queues without Explicit State Information Sharing
Limiting the policy to this class reduces the problem to obtaining a queue switching policy at queue emptiness instants.
A bandit approach to curriculum generation for automatic speech recognition
The Automated Speech Recognition (ASR) task has been a challenging domain especially for low data scenarios with few audio examples.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
SAGRNN: Self-Attentive Gated RNN for Binaural Speaker Separation with Interaural Cue Preservation
In addition, our approach effectively preserves the interaural cues, which improves the accuracy of sound localization.
Audio and Speech Processing Sound
A Sequential Self Teaching Approach for Improving Generalization in Sound Event Recognition
An important problem in machine auditory perception is to recognize and detect sound events.
Ranked #31 on
Audio Classification
on AudioSet
Large Scale Audiovisual Learning of Sounds with Weakly Labeled Data
Recognizing sounds is a key aspect of computational audio scene analysis and machine perception.
Ranked #22 on
Audio Classification
on AudioSet
Secost: Sequential co-supervision for large scale weakly labeled audio event detection
Weakly supervised learning algorithms are critical for scaling audio event detection to several hundreds of sound categories.
Learning Sound Events From Webly Labeled Data
In the last couple of years, weakly labeled learning for sound events has turned out to be an exciting approach for audio event detection.
A Closer Look at Weak Label Learning for Audio Events
In this work, we first describe a CNN based approach for weakly supervised training of audio events.
NELS-Never-Ending Learner of Sounds
Sounds are essential to how humans perceive and interact with the world.
Knowledge Transfer from Weakly Labeled Audio using Convolutional Neural Network for Sound Events and Scenes
In this work we propose approaches to effectively transfer knowledge from weakly labeled web audio data.
Sound Multimedia Audio and Speech Processing
Framework for evaluation of sound event detection in web videos
The framework crawls videos using search queries corresponding to 78 sound event labels drawn from three datasets.
Deep CNN Framework for Audio Event Recognition using Weakly Labeled Web Data
We propose that learning algorithms that can exploit weak labels offer an effective method to learn from web data.
Audio Event and Scene Recognition: A Unified Approach using Strongly and Weakly Labeled Data
In this paper we propose a novel learning framework called Supervised and Weakly Supervised Learning where the goal is to learn simultaneously from weakly and strongly labeled data.
Discovering Sound Concepts and Acoustic Relations In Text
In this paper we describe approaches for discovering acoustic concepts and relations in text.
An Approach for Self-Training Audio Event Detectors Using Web Data
The audio event detectors are trained on the labeled audio and ran on the unlabeled audio downloaded from YouTube.
Features and Kernels for Audio Event Recognition
One of the most important problems in audio event detection research is absence of benchmark results for comparison with any proposed method.
Sound Multimedia
Classifier Risk Estimation under Limited Labeling Resources
In this paper we propose strategies for estimating performance of a classifier when labels cannot be obtained for the whole test set.
Weakly Supervised Scalable Audio Content Analysis
Audio Event Detection is an important task for content analysis of multimedia data.
Speech Enhancement In Multiple-Noise Conditions using Deep Neural Networks
In this paper we consider the problem of speech enhancement in real-world like conditions where multiple noises can simultaneously corrupt speech.
Sound
Audio Event Detection using Weakly Labeled Data
This helps in obtaining a complete description of the recording and is notable since temporal information was never known in the first place in weakly labeled data.
Unsupervised Fusion Weight Learning in Multiple Classifier Systems
We also introduce a novel metric for ranking instances based on an index which depends upon the rank of weighted scores of test points among the weighted scores of training points.
