Search Results for author:
Found 47 papers, 18 papers with code
$\text{R}^2$-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations
Referring perception, which aims at grounding visual objects with multimodal referring guidance, is essential for bridging the gap between humans, who provide instructions, and the environment where intelligent systems perceive.
A General Framework for Learning from Weak Supervision
Weakly supervised learning generally faces challenges in applicability to various scenarios with diverse weak supervision and in scalability due to the complexity of existing algorithms, thereby hindering the practical deployment.
PAM: Prompting Audio-Language Models for Audio Quality Assessment
Here, we exploit this capability and introduce PAM, a no-reference metric for assessing audio quality for different audio processing tasks.
Token Prediction as Implicit Classification to Identify LLM-Generated Text
This paper introduces a novel approach for identifying the possible large language models (LLMs) involved in text generation.
Pairwise Similarity Learning is SimPLE
In this paper, we focus on a general yet important learning problem, pairwise similarity learning (PSL).
Prompting Audios Using Acoustic Properties For Emotion Representation
In this work, we address the challenge of automatically generating these prompts and training a model to better learn emotion representations from audio and prompt pairs.
LoFT: Local Proxy Fine-tuning For Improving Transferability Of Adversarial Attacks Against Large Language Model
We hypothesize that for attacks to be transferrable, it is sufficient if the proxy can approximate the target model in the neighborhood of the harmful query.
Evaluating Speech Synthesis by Training Recognizers on Synthetic Speech
In this paper, we propose an evaluation technique involving the training of an ASR model on synthetic speech and assessing its performance on real speech.
Completing Visual Objects via Bridging Generation and Segmentation
This paper presents a novel approach to object completion, with the primary goal of reconstructing a complete object from its partially visible components.
Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decomposition
We propose a semantic decomposition method based on product quantization, where the multi-source semantics can be decomposed and represented by several disentangled and noise-suppressed single-source semantics.
Importance of negative sampling in weak label learning
Weak-label learning is a challenging task that requires learning from data "bags" containing positive and negative instances, but only the bag labels are known.
Training Audio Captioning Models without Audio
During inference, the text encoder is replaced with the pretrained CLAP audio encoder.
Rethinking Voice-Face Correlation: A Geometry View
Previous works on voice-face matching and voice-guided face synthesis demonstrate strong correlations between voice and face, but mainly rely on coarse semantic cues such as gender, age, and emotion.
The Hidden Dance of Phonemes and Visage: Unveiling the Enigmatic Link between Phonemes and Facial Features
This work unveils the enigmatic link between phonemes and facial features.
BASS: Block-wise Adaptation for Speech Summarization
End-to-end speech summarization has been shown to improve performance over cascade baselines.
Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations
In this paper, we introduce imprecise label learning (ILL), a framework for the unification of learning with various imprecise label configurations.
Ranked #1 on
Learning with noisy labels
on mini WebVision 1.0
Pengi: An Audio Language Model for Audio Tasks
We introduce Pengi, a novel Audio Language Model that leverages Transfer Learning by framing all audio tasks as text-generation tasks.
GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content
This paper presents a novel approach for detecting ChatGPT-generated vs. human-written text using language models.
Describing emotions with acoustic property prompts for speech emotion recognition
We investigate how the model can learn to associate the audio with the descriptions, resulting in performance improvement of Speech Emotion Recognition and Speech Audio Retrieval.
Unifying the Discrete and Continuous Emotion labels for Speech Emotion Recognition
Accordingly, models that have been proposed for emotion detection use one or the other of these label types.
Self-supervision and Learnable STRFs for Age, Emotion, and Country Prediction
This work presents a multitask approach to the simultaneous estimation of age, country of origin, and emotion given vocal burst audio for the 2022 ICML Expressive Vocalizations Challenge ExVo-MultiTask track.
On the pragmatism of using binary classifiers over data intensive neural network classifiers for detection of COVID-19 from voice
Lately, there has been a global effort by multiple research groups to detect COVID-19 from voice.
An Overview of Techniques for Biomarker Discovery in Voice Signal
This paper reflects on the effect of several categories of medical conditions on human voice, focusing on those that may be hypothesized to have effects on voice, but for which the changes themselves may be subtle enough to have eluded observation in standard analytical examinations of the voice signal.
Self-Supervised 3D Face Reconstruction via Conditional Estimation
We present a conditional estimation (CEST) framework to learn 3D facial parameters from 2D single-view images by self-supervised training from videos.
Ranked #15 on
3D Face Reconstruction
on REALY
SphereFace Revived: Unifying Hyperspherical Face Recognition
As one of the earliest works in hyperspherical face recognition, SphereFace explicitly proposed to learn face embeddings with large inter-class angular margin.
SphereFace2: Binary Classification is All You Need for Deep Face Recognition
In this paper, we start by identifying the discrepancy between training and evaluation in the existing multi-class classification framework and then discuss the potential limitations caused by the "competitive" nature of softmax normalization.
Controlled AutoEncoders to Generate Faces from Voices
With this in perspective, we propose a framework to morph a target face in response to a given voice in a way that facial features are implicitly guided by learned voice-face correlation in this paper.
Improving weakly supervised sound event detection with self-supervised auxiliary tasks
To that extent, we propose a shared encoder architecture with sound event detection as a primary task and an additional secondary decoder for a self-supervised auxiliary task.
Masked Proxy Loss For Text-Independent Speaker Verification
We further propose Multinomial Masked Proxy (MMP) loss to leverage the hardness of speaker pairs.
Interpreting glottal flow dynamics for detecting COVID-19 from voice
In the pathogenesis of COVID-19, impairment of respiratory functions is often one of the key symptoms.
Detection of COVID-19 through the analysis of vocal fold oscillations
Phonation, or the vibration of the vocal folds, is the primary source of vocalization in the production of voiced sounds by humans.
Multi-Task Learning for Interpretable Weakly Labelled Sound Event Detection
Weakly Labelled learning has garnered lot of attention in recent years due to its potential to scale Sound Event Detection (SED) and is formulated as Multiple Instance Learning (MIL) problem.
Face Reconstruction from Voice using Generative Adversarial Networks
The network learns to generate faces from voices by matching the identities of generated faces to those of the speakers, on a training set.
The phonetic bases of vocal expressed emotion: natural versus acted
Our tests show significant differences in the manner and choice of phonemes in acted and natural speech, concluding moderate to low validity and value in using acted speech databases for emotion classification tasks.
Detecting gender differences in perception of emotion in crowdsourced data
While we limit ourselves to a single modality (i. e. speech), our framework is applicable to studies of emotion perception from all such loosely annotated data in general.
Non-Determinism in Neural Networks for Adversarial Robustness
Recent breakthroughs in the field of deep learning have led to advancements in a broad spectrum of tasks in computer vision, audio processing, natural language processing and other areas.
Reconstructing faces from voices
Voice profiling aims at inferring various human parameters from their speech, e. g. gender, age, etc.
Hierarchical Routing Mixture of Experts
Addressing these problems, we propose a binary tree-structured hierarchical routing mixture of experts (HRME) model that has classifiers as non-leaf node experts and simple regression models as leaf node experts.
Hide and Speak: Towards Deep Neural Networks for Speech Steganography
Steganography is the science of hiding a secret message within an ordinary public message, which is referred to as Carrier.
Neural Regression Trees
Regression-via-Classification (RvC) is the process of converting a regression problem to a classification one.
Neural Regression Tree
Regression-via-Classification (RvC) is the process of converting a regression problem to a classification one.
Disjoint Mapping Network for Cross-modal Matching of Voices and Faces
We propose a novel framework, called Disjoint Mapping Network (DIMNet), for cross-modal biometric matching, in particular of voices and faces.
Optimal Strategies for Matching and Retrieval Problems by Comparing Covariates
In many retrieval problems, where we must retrieve one or more entries from a gallery in response to a probe, it is common practice to learn to do by directly comparing the probe and gallery entries to one another.
Voice Impersonation using Generative Adversarial Networks
In voice impersonation, the resultant voice must convincingly convey the impression of having been naturally produced by the target speaker, mimicking not only the pitch and other perceivable signal qualities, but also the style of the target speaker.
Sound Audio and Speech Processing
Speaker identification from the sound of the human breath
The goal of this paper is to demonstrate that breath sounds are indeed bio-signatures that can be used to identify speakers.
Content-based Video Indexing and Retrieval Using Corr-LDA
Existing video indexing and retrieval methods on popular web-based multimedia sharing websites are based on user-provided sparse tagging.
Plagiarism Detection in Polyphonic Music using Monaural Signal Separation
Given the large number of new musical tracks released each year, automated approaches to plagiarism detection are essential to help us track potential violations of copyright.
