Search Results for author:
Found 37 papers, 28 papers with code
One-class learning towards generalized voice spoofing detection
Human voices can be used to authenticate the identity of the speaker, but the automatic speaker verification (ASV) systems are vulnerable to voice spoofing attacks, such as impersonation, replay, text-to-speech, and voice conversion.
Don't look back: an online beat tracking method using RNN and enhanced particle filtering
Most preexisting OBT methods either apply some offline approaches to a moving window containing past data to make predictions about future beat positions or must be primed with past data at startup to initialize.
BeatNet: CRNN and Particle Filtering for Online Joint Beat Downbeat and Meter Tracking
The online estimation of rhythmic information, such as beat positions, downbeat positions, and meter, is critical for many real-time music applications.
Ranked #1 on
Online Beat Tracking
on Rock Corpus
Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss
We devise a cascade GAN approach to generate talking face video, which is robust to different face shapes, view angles, facial characteristics, and noisy audio conditions.
Audio-Visual Event Localization in Unconstrained Videos
In this paper, we introduce a novel problem of audio-visual event localization in unconstrained videos.
Speech Driven Talking Face Generation from a Single Image and an Emotion Condition
Visual emotion expression plays an important role in audiovisual speech communication.
ControlVC: Zero-Shot Voice Conversion with Time-Varying Controls on Pitch and Speed
In this paper, we propose ControlVC, the first neural voice conversion system that achieves time-varying controls on pitch and speed.
An Empirical Study on Channel Effects for Synthetic Voice Spoofing Countermeasure Systems
Spoofing countermeasure (CM) systems are critical in speaker verification; they aim to discern spoofing attacks from bona fide speech trials.
UR Channel-Robust Synthetic Speech Detection System for ASVspoof 2021
Different from previous ASVspoof challenges, the LA task this year presents codec and transmission channel variability, while the new task DF presents general audio compression.
Skipping the Frame-Level: Event-Based Piano Transcription With Neural Semi-CRFs
When formulating piano transcription in this way, we eliminate the need to rely on disjoint frame-level estimates for different stages of a note event.
A Novel 1D State Space for Efficient Music Rhythmic Analysis
Inferring music time structures has a broad range of applications in music production, processing and analysis.
Ranked #1 on
Online Beat Tracking
on GTZAN
SingFake: Singing Voice Deepfake Detection
These unique properties make singing voice deepfake detection a relevant but significantly different problem from synthetic speech detection.
Lip Movements Generation at a Glance
In this paper, we consider a task of such: given an arbitrary audio speech and one lip image of arbitrary target identity, generate synthesized lip movements of the target identity saying the speech.
SAMO: Speaker Attractor Multi-Center One-Class Learning for Voice Anti-Spoofing
Our previous research on one-class learning has improved the generalization ability to unseen attacks by compacting the bona fide speech in the embedding space.
Y-Vector: Multiscale Waveform Encoder for Speaker Embedding
State-of-the-art text-independent speaker verification systems typically use cepstral features or filter bank energies as speech features.
Singing Beat Tracking With Self-supervised Front-end and Linear Transformers
Tracking beats of singing voices without the presence of musical accompaniment can find many applications in music production, automatic song arrangement, and social media interaction.
Learning Sparse Analytic Filters for Piano Transcription
In this work, several variations of a frontend filterbank learning module are investigated for piano transcription, a challenging low-level music information retrieval task.
Music Source Separation with Generative Flow
Fully-supervised models for source separation are trained on parallel mixture-source data and are currently state-of-the-art.
HRTF Field: Unifying Measured HRTF Magnitude Representation with Neural Fields
In this work, we propose to use neural fields, a differentiable representation of functions through neural networks, to model HRTFs with arbitrary spatial sampling schemes.
Mitigating Cross-Database Differences for Learning Unified HRTF Representation
We further show that these normalized HRTFs can be used to learn a more unified HRTF representation across databases than the prior art.
A Data-Driven Methodology for Considering Feasibility and Pairwise Likelihood in Deep Learning Based Guitar Tablature Transcription Systems
This naturally enforces playability constraints for guitar, and yields tablature which is more consistent with the symbolic data used to estimate pairwise likelihoods.
A Probabilistic Fusion Framework for Spoofing Aware Speaker Verification
We further propose fusion strategies for direct inference and fine-tuning to predict the SASV score based on the framework.
When Counterpoint Meets Chinese Folk Melodies
An interaction reward model is trained on the duets formed from outer parts of Bach chorales to model counterpoint interaction, while a style reward model is trained on monophonic melodies of Chinese folk songs to model melodic patterns.
Transcription free filler word detection with Neural semi-CRFs
Non-linguistic filler words, such as "uh" or "um", are prevalent in spontaneous speech and serve as indicators for expressing hesitation or uncertainty.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Predicting Global Head-Related Transfer Functions From Scanned Head Geometry Using Deep Learning and Compact Representations
For the HRTF data, we use truncated spherical harmonic (SH) coefficients to represent the HRTF magnitudes and onsets.
A study of the robustness of raw waveform based speaker embeddings under mismatched conditions
In this paper, we conduct a cross-dataset study on parametric and non-parametric raw-waveform based speaker embeddings through speaker verification experiments.
Learning Arousal-Valence Representation from Categorical Emotion Labels of Speech
In this work, we propose to learn the AV representation from categorical emotion labels of speech.
Toward Fully Self-Supervised Multi-Pitch Estimation
Multi-pitch estimation is a decades-long research problem involving the detection of pitch activity associated with concurrent musical events within multi-instrument mixtures.
Generating Talking Face Landmarks from Speech
In this paper, we present a system that can generate landmark points of a talking face from an acoustic speech in real time.
Deep Cross-Modal Audio-Visual Generation
Being the first to explore this new problem, we compose two new datasets with pairs of images and sounds of musical performances of different instruments.
RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning
We cast this as a reinforcement learning problem, where the generation agent learns a policy to generate a musical note (action) based on previously generated context (state).
Themes Informed Audio-visual Correspondence Learning
The applications of short-term user-generated video (UGV), such as Snapchat, and Youtube short-term videos, booms recently, raising lots of multimodal machine learning tasks.
Audiovisual Singing Voice Separation
Separating a song into vocal and accompaniment components is an active research topic, and recent years witnessed an increased performance from supervised training using deep learning techniques.
Rethinking Audio-visual Synchronization for Active Speaker Detection
This clarification of definition is motivated by our extensive experiments, through which we discover that existing ASD methods fail in modeling the audio-visual synchronization and often classify unsynchronized videos as active speaking.
SingNet: A Real-time Singing Voice Beat and Downbeat Tracking System
Singing voice beat and downbeat tracking posses several applications in automatic music production, analysis and manipulation.
SynthTab: Leveraging Synthesized Data for Guitar Tablature Transcription
Guitar Tablature Transcription (GTT) is an important task with broad applications in music education, composition, and entertainment.
Scoring Intervals using Non-Hierarchical Transformer For Automatic Piano Transcription
The neural semi-Markov Conditional Random Field (semi-CRF) framework has demonstrated promise for event-based piano transcription.
