Search Results for author:
Found 19 papers, 5 papers with code
HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances
To generate images with realistic hands, we propose a novel diffusion-based architecture called HanDiffuser that achieves realism by injecting hand embeddings in the generative process.
VaQuitA: Enhancing Alignment in LLM-Assisted Video Understanding
Recent advancements in language-model-based video understanding have been progressing at a remarkable pace, spurred by the introduction of Large Language Models (LLMs).
Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior
We call these models Large Content and Behavior Models (LCBMs).
Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention
We propose a method to detect individualized highlights for users on given target videos based on their preferred highlight clips marked on previous videos they have watched.
HighlightMe: Detecting Highlights from Human-Centric Videos
We train our network to map the activity- and interaction-based latent structural representations of the different modalities to per-frame highlight scores based on the representativeness of the frames.
Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning
Our network consists of two components: a generator to synthesize gestures from a joint embedding space of features encoded from the input speech and the seed poses, and a discriminator to distinguish between the synthesized pose sequences and real 3D pose sequences.
Ranked #4 on
Gesture Generation
on TED Gesture Dataset
Learning Unseen Emotions from Gestures via Semantically-Conditioned Zero-Shot Perception with Adversarial Autoencoders
Our task is to map gestures to novel emotion categories not encountered in training.
EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle
We report an AP of 65. 83 across 4 categories on GroupWalk, which is also an improvement over prior methods.
Ranked #2 on
Emotion Recognition in Context
on CAER
Emotion Recognition in Context
Multimodal Emotion Recognition
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues
Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake".
The Liar's Walk: Detecting Deception with Gait and Gesture
We present a data-driven deep neural algorithm for detecting deceptive walking behavior using nonverbal cues like gaits and gestures.
Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs
In practice, our approach reduces the average prediction error by more than 54% over prior algorithms and achieves a weighted average accuracy of 91. 2% for behavior prediction.
Ranked #1 on
Trajectory Prediction
on ApolloScape
Robotics
Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping
For the annotated data, we also train a classifier to map the latent embeddings to emotion labels.
M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues
Our approach combines cues from multiple co-occurring modalities (such as face, text, and speech) and also is more robust than other methods to sensor noise in any of the individual modalities.
STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE).
RobustTP: End-to-End Trajectory Prediction for Heterogeneous Road-Agents in Dense Traffic with Noisy Sensor Inputs
RobustTP is an approach that first computes trajectories using a combination of a non-linear motion model and a deep learning-based instance segmentation algorithm.
Robotics
RoadTrack: Realtime Tracking of Road Agents in Dense and Heterogeneous Environments
We present a realtime tracking algorithm, RoadTrack, to track heterogeneous road-agents in dense traffic videos.
Robotics
Identifying Emotions from Walking using Affective and Deep Features
We also present an EWalk (Emotion Walk) dataset that consists of videos of walking individuals with gaits and labeled emotions.
Efficient and Robust Registration on the 3D Special Euclidean Group
Our approach significantly outperforms the state-of-the-art robust 3D registration method based on a line process in terms of both speed and accuracy.
TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions
We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories.
Ranked #1 on
Trajectory Prediction
on TRAF
Trajectory Prediction
Robotics
