Search Results for author:
Found 20 papers, 8 papers with code
Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation
In this paper, we present a novel data simulation pipeline that produces diverse training data from a range of acoustic environments and content, and propose new training paradigms to improve quality of a general speech separation model.
SpeechCraft: A Fine-grained Expressive Speech Dataset with Natural Language Description
To tackle this challenge, we propose an automatic speech annotation system for expressiveness interpretation that annotates in-the-wild speech clips with expressive and vivid human language descriptions.
VDGD: Mitigating LVLM Hallucinations in Cognitive Prompts by Bridging the Visual Perception Gap
From our analysis, we show that: (1) The community's efforts have been primarily targeted towards reducing hallucinations related to visual recognition (VR) prompts (e. g., prompts that only require describing the image), thereby ignoring hallucinations for cognitive prompts (e. g., prompts that require additional skills like reasoning on contents of the image).
A Closer Look at the Limitations of Instruction Tuning
Our findings reveal that responses generated solely from pre-trained knowledge consistently outperform responses by models that learn any form of new knowledge from IT on open-source datasets.
Efficient Spoken Language Recognition via Multilabel Classification
Spoken language recognition (SLR) is the task of automatically identifying the language present in a speech signal.
Audio Similarity is Unreliable as a Proxy for Audio Quality
Many audio processing tasks require perceptual assessment.
Music Enhancement via Image Translation and Vocoding
Consumer-grade music recordings such as those captured by mobile devices typically contain distortions in the form of background noise, reverb, and microphone-induced EQ.
HEAR: Holistic Evaluation of Audio Representations
The aim of the HEAR benchmark is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios.
Neural Pitch-Shifting and Time-Stretching with Controllable LPCNet
Modifying the pitch and timing of an audio signal are fundamental audio editing operations with applications in speech manipulation, audio-visual synchronization, and singing voice editing and synthesis.
Controllable deep melody generation via hierarchical music structure representation
Recent advances in deep learning have expanded possibilities to generate music, but generating a customizable full piece of music with consistent long-term structure remains a challenge.
Context-Aware Prosody Correction for Text-Based Speech Editing
Text-based speech editors expedite the process of editing speech recordings by permitting editing via intuitive cut, copy, and paste operations on a speech transcript.
CDPAM: Contrastive learning for perceptual audio similarity
The DPAM approach of Manocha et al. learns a full-reference metric trained directly on human judgments, and thus correlates well with human perception.
Disentangled Multidimensional Metric Learning for Music Similarity
For this task, it is typically necessary to define a similarity metric to compare one recording to another.
Metric Learning vs Classification for Disentangled Music Representation Learning
For this, we (1) outline past work on the relationship between metric learning and classification, (2) extend this relationship to multi-label data by exploring three different learning approaches and their disentangled versions, and (3) evaluate all models on four tasks (training time, similarity retrieval, auto-tagging, and triplet prediction).
Controllable Neural Prosody Synthesis
Speech synthesis has recently seen significant improvements in fidelity, driven by the advent of neural vocoders and neural prosody generators.
HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks
Real-world audio recordings are often degraded by factors such as noise, reverberation, and equalization distortion.
F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder
Recently, AutoVC, a conditional autoencoders (CAEs) based method achieved state-of-the-art results by disentangling the speaker identity and speech content using information-constraining bottlenecks, and it achieves zero-shot conversion by swapping in a different speaker's identity embedding to synthesize a new voice.
A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences
Assessment of many audio processing tasks relies on subjective evaluation which is time-consuming and expensive.
Text-based Editing of Talking-head Video
To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material.
Perceptually-motivated Environment-specific Speech Enhancement
This paper introduces a deep learning approach to enhance speech recordings made in a specific environment.
