Search Results for author:
Found 21 papers, 10 papers with code
Joint Estimation and Analysis of Risk Behavior Ratings in Movie Scripts
Exposure to violent, sexual, or substance-abuse content in media increases the willingness of children and adolescents to imitate similar behaviors.
A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation
Here, we propose a novel training approach to effectively learn arbitrary conditional distributions in the audiovisual space. Our key contribution lies in how we parameterize the diffusion timestep in the forward diffusion process.
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #4 on
Text-to-Video Generation
on MSR-VTT
LanSER: Language-Model Supported Speech Emotion Recognition
Speech emotion recognition (SER) models typically rely on costly human-labeled data for training, making scaling methods to large speech datasets and nuanced emotion taxonomies difficult.
MM-AU:Towards Multimodal Understanding of Advertisement Videos
Advertisement videos (ads) play an integral part in the domain of Internet e-commerce as they amplify the reach of particular products to a broad audience or can serve as a medium to raise awareness about specific issues through concise narrative structures.
Contextually-rich human affect perception using multimodal scene information
The process of human affect understanding involves the ability to infer person specific emotional states from various sources including images, speech, and language.
Heterogeneous Graph Learning for Acoustic Event Classification
Heterogeneous graphs provide a compact, efficient, and scalable way to model data involving multiple disparate modalities.
A dataset for Audio-Visual Sound Event Detection in Movies
In this work, we present a dataset of audio events called Subtitle-Aligned Movie Sounds (SAM-S).
MovieCLIP: Visual Scene Recognition in Movies
Longform media such as movies have complex narrative structures, with events spanning a rich variety of ambient visual scenes.
Visually-aware Acoustic Event Detection using Heterogeneous Graphs
In contrast, we employ heterogeneous graphs to explicitly capture the spatial and temporal relationships between the modalities and represent detailed information about the underlying signal.
Multitask vocal burst modeling with ResNets and pre-trained paralinguistic Conformers
This technical report presents the modeling approaches used in our submission to the ICML Expressive Vocalizations Workshop & Competition multitask track (ExVo-MultiTask).
To train or not to train adversarially: A study of bias mitigation strategies for speaker recognition
In this paper, we systematically evaluate the biases present in speaker recognition systems with respect to gender across a range of system operating points.
Self-supervised Graphs for Audio Representation Learning with Limited Labeled Data
Large scale databases with high-quality manual annotations are scarce in audio domain.
Understanding of Emotion Perception from Art
Computational modeling of the emotions evoked by art in humans is a challenging problem because of the subjective and nuanced nature of art and affective signals.
Representation of professions in entertainment media: Insights into frequency and sentiment trends through computational text analysis
We analyze the frequency and sentiment trends of different occupations, study the effect of media attributes like genre, country of production, and title type on these trends, and investigate if the incidence of professions in media subtitles correlate with their real-world employment statistics.
Robust Character Labeling in Movie Videos: Data Resources and Self-supervised Feature Adaptation
Our work in this paper focuses on two key aspects of this problem: the lack of domain-specific training or benchmark datasets, and adapting face embeddings learned on web images to long-form content, specifically movies.
Victim or Perpetrator? Analysis of Violent Characters Portrayals from Movie Scripts
To date, we are the first to show that language used in movie scripts is a strong indicator of violent content, and that there are systematic portrayals of certain demographics as victims and perpetrators in a large dataset.
Generalized Multi-view Shared Subspace Learning using View Bootstrapping
A key objective in multi-view learning is to model the information common to multiple parallel views of a class of objects/events to improve downstream learning tasks.
Cross modal video representations for weakly supervised active speaker localization
Avoiding the need for manual annotations for active speakers in visual frames, acquiring of which is very expensive, we present a weakly supervised system for the task of localizing active speakers in movie content.
Robust speaker recognition using unsupervised adversarial invariance
In this paper, we address the problem of speaker recognition in challenging acoustic conditions using a novel method to extract robust speaker-discriminative speech representations.
Multimodal Representation Learning using Deep Multiset Canonical Correlation
We propose Deep Multiset Canonical Correlation Analysis (dMCCA) as an extension to representation learning using CCA when the underlying signal is observed across multiple (more than two) modalities.
