Search Results for author:
Found 59 papers, 12 papers with code
FlashSLAM: Accelerated RGB-D SLAM for Real-Time 3D Scene Reconstruction with Gaussian Splatting
We present FlashSLAM, a novel SLAM approach that leverages 3D Gaussian Splatting for efficient and robust 3D scene reconstruction.
Zonal RL-RRT: Integrated RL-RRT Path Planning with Collision Probability and Zone Connectivity
Path planning in high-dimensional spaces poses significant challenges, particularly in achieving both time efficiency and a fair success rate.
EfficientEQA: An Efficient Approach for Open Vocabulary Embodied Question Answering
Embodied Question Answering (EQA) is an essential yet challenging task for robotic home assistants.
COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Models
We propose a novel framework COLLAGE for generating collaborative agent-object-agent interactions by leveraging large language models (LLMs) and hierarchical motion-specific vector-quantized variational autoencoders (VQ-VAEs).
SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models
We demonstrate the effectiveness and generalizability of SELP across different robot agents and tasks, including drone navigation and robot manipulation.
Go-SLAM: Grounded Object Segmentation and Localization with Gaussian Splatting SLAM
We introduce Go-SLAM, a novel framework that utilizes 3D Gaussian Splatting SLAM to reconstruct dynamic environments while embedding object-level information within the scene representations.
Dynamic Obstacle Avoidance through Uncertainty-Based Adaptive Planning with Diffusion
By framing reinforcement learning as a sequence modeling problem, recent work has enabled the use of generative models, such as diffusion models, for planning.
Multi-Robot Informative Path Planning for Efficient Target Mapping using Deep Reinforcement Learning
We propose a novel deep reinforcement learning approach for multi-robot informative path planning to map targets-of-interest in an unknown 3D environment.
MVGaussian: High-Fidelity text-to-3D Content Generation with Multi-View Guidance and Surface Densification
The field of text-to-3D content generation has made significant progress in generating realistic 3D objects, with existing methodologies like Score Distillation Sampling (SDS) offering promising guidance.
Adaptive Planning with Generative Models under Uncertainty
Planning with generative models has emerged as an effective decision-making paradigm across a wide range of domains, including reinforcement learning and autonomous navigation.
Reduced-Order Neural Operators: Learning Lagrangian Dynamics on Highly Sparse Graphs
We propose accelerating the simulation of Lagrangian dynamics, such as fluid flows, granular flows, and elastoplasticity, with neural-operator-based reduced-order modeling.
Speech2UnifiedExpressions: Synchronous Synthesis of Co-Speech Affective Face and Body Expressions from Affordable Inputs
To enhance the plausibility of synthesis, we use an adversarial discriminator that learns to differentiate between the face and pose motions computed from the original videos and our synthesized motions based on their affective expressions.
Quantifying Uncertainty in Motion Prediction with Variational Bayesian Mixture
In this paper, we propose the Sequential Neural Variational Agent (SeNeVA), a generative model that describes the distribution of future trajectories for a single moving object.
Beyond Text: Utilizing Vocal Cues to Improve Decision Making in LLMs for Robot Navigation Tasks
While LLMs excel in processing text in these human conversations, they struggle with the nuances of verbal instructions in scenarios like social navigation, where ambiguity and uncertainty can erode trust in robotic and other AI systems.
DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision
We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS).
RL Dreams: Policy Gradient Optimization for Score Distillation based 3D Generation
Further, the recent work of Denoising Diffusion Policy Optimization (DDPO) demonstrates that the diffusion process is compatible with policy gradient methods and has been demonstrated to improve the 2D diffusion models using an aesthetic scoring function.
LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs
Autonomous driving (AD) has made significant strides in recent years.
MIRACLE: Inverse Reinforcement and Curriculum Learning Model for Human-inspired Mobile Robot Navigation
In emergency scenarios, mobile robots must navigate like humans, interpreting stimuli to locate potential victims rapidly without interfering with first responders.
MTAC: Hierarchical Reinforcement Learning-based Multi-gait Terrain-adaptive Quadruped Controller
Urban search and rescue missions require rapid first response to minimize loss of life and damage.
DREAM: Decentralized Reinforcement Learning for Exploration and Efficient Energy Management in Multi-Robot Systems
This paper introduces DREAM - Decentralized Reinforcement Learning for Exploration and Efficient Energy Management in Multi-Robot Systems, a comprehensive framework that optimizes the allocation of resources for efficient exploration.
Optimizing Crowd-Aware Multi-Agent Path Finding through Local Communication with Graph Neural Networks
Multi-Agent Path Finding (MAPF) in crowded environments presents a challenging problem in motion planning, aiming to find collision-free paths for all agents in the system.
DF-TransFusion: Multimodal Deepfake Detection via Lip-Audio Cross-Attention and Facial Self-Attention
With the rise in manipulated media, deepfake detection has become an imperative task for preserving the authenticity of digital content.
AffectEcho: Speaker Independent and Language-Agnostic Emotion and Affect Transfer for Speech Synthesis
We showcase the language-independent emotion modeling capability of the quantized emotional embeddings learned from a bilingual (English and Chinese) speech corpus with an emotion transfer task from a reference speech to a target speech.
Principles and Guidelines for Evaluating Social Robot Navigation Algorithms
A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation.
SG-LSTM: Social Group LSTM for Robot Navigation Through Dense Crowds
Our research aims to develop a model that can predict the movements of pedestrians and perceptually-social groups in crowded environments.
DroNeRF: Real-time Multi-agent Drone Pose Optimization for Computing Neural Radiance Fields
Using drones in conjunction with NeRF provides a unique and dynamic way to generate novel views of a scene, especially with limited scene capabilities of restricted movements.
AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning
We propose a novel approach for aerial video action recognition.
Ranked #1 on
Action Recognition
on RoCoG-v2
Neural Operator: Is data all you need to model the world? An insight into the impact of Physics Informed Machine Learning
This article aims to provide a comprehensive insight into how data-driven approaches can complement conventional techniques to solve engineering and physics problems, while also noting some of the major pitfalls of machine learning-based approaches.
PaCMO: Partner Dependent Human Motion Generation in Dyadic Human Activity using Neural Operators
In contrast to the concurrent works, which mainly focus on generating the motion of a single actor from the textual description, we generate the motion of one of the actors from the motion of the other participating actor in the action.
AdaFNIO: Adaptive Fourier Neural Interpolation Operator for video frame interpolation
We present, AdaFNIO - Adaptive Fourier Neural Interpolation Operator, a neural operator-based architecture to perform video frame interpolation.
Placing Human Animations into 3D Scenes by Learning Interaction- and Geometry-Driven Keyframes
We compare our method, which we call PAAK, with prior approaches, including POSA, PROX ground truth, and a motion synthesis method, and highlight the benefits of our method with a perceptual study.
3MASSIV: Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videos
We present 3MASSIV, a multilingual, multimodal and multi-aspect, expertly-annotated dataset of diverse short videos extracted from short-video social media platform - Moj.
Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality
We use an LSTM-based learning model for emotion perception.
SeekNet: Improved Human Instance Segmentation and Tracking via Reinforcement Learning Based Optimized Robot Relocation
Amodal recognition is the ability of the system to detect occluded objects.
RAIST: Learning Risk Aware Traffic Interactions via Spatio-Temporal Graph Convolutional Networks
A key aspect of driving a road vehicle is to interact with other road users, assess their intentions and make risk-aware tactical decisions.
Developing an Effective and Automated Patient Engagement Estimator for Telehealth: A Machine Learning Approach
We discuss MET, a learning-based algorithm proposed for perceiving a patient's level of engagement during telehealth sessions.
Can a Robot Trust You? A DRL-Based Approach to Trust-Driven Human-Guided Navigation
We utilize both these trust metrics into an optimal cognitive reasoning scheme that decides when and when not to trust the given guidance.
Learning Unseen Emotions from Gestures via Semantically-Conditioned Zero-Shot Perception with Adversarial Autoencoders
Our task is to map gestures to novel emotion categories not encountered in training.
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues
Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake".
EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle
We report an AP of 65. 83 across 4 categories on GroupWalk, which is also an improvement over prior methods.
Ranked #2 on
Emotion Recognition in Context
on CAER
Emotion Recognition in Context
Multimodal Emotion Recognition
ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation
Our approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints.
Ranked #1 on
Emotion Classification
on EWALK
The Liar's Walk: Detecting Deception with Gait and Gesture
We present a data-driven deep neural algorithm for detecting deceptive walking behavior using nonverbal cues like gaits and gestures.
Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs
In practice, our approach reduces the average prediction error by more than 54% over prior algorithms and achieves a weighted average accuracy of 91. 2% for behavior prediction.
Ranked #1 on
Trajectory Prediction
on ApolloScape
Robotics
Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping
For the annotated data, we also train a classifier to map the latent embeddings to emotion labels.
M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues
Our approach combines cues from multiple co-occurring modalities (such as face, text, and speech) and also is more robust than other methods to sensor noise in any of the individual modalities.
STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE).
RobustTP: End-to-End Trajectory Prediction for Heterogeneous Road-Agents in Dense Traffic with Noisy Sensor Inputs
RobustTP is an approach that first computes trajectories using a combination of a non-linear motion model and a deep learning-based instance segmentation algorithm.
Robotics
FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics
We also investigate the perception of a user in an AR setting and observe that an FVA has a statistically significant improvement in terms of the perceived friendliness and social presence of a user compared to an agent without the friendliness modeling.
RoadTrack: Realtime Tracking of Road Agents in Dense and Heterogeneous Environments
We present a realtime tracking algorithm, RoadTrack, to track heterogeneous road-agents in dense traffic videos.
Robotics
Identifying Emotions from Walking using Affective and Deep Features
We also present an EWalk (Emotion Walk) dataset that consists of videos of walking individuals with gaits and labeled emotions.
TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions
We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories.
Ranked #1 on
Trajectory Prediction
on TRAF
Trajectory Prediction
Robotics
Pedestrian Dominance Modeling for Socially-Aware Robot Navigation
We present a Pedestrian Dominance Model (PDM) to identify the dominance characteristics of pedestrians for robot navigation.
Robotics
Data-Driven Modeling of Group Entitativity in Virtual Environments
We also present a novel interactive multi-agent simulation algorithm to model entitative groups and conduct a VR user study to validate the socio-emotional predictive power of our algorithm.
Graphics Human-Computer Interaction
PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians
Our planning system combines a POMDP algorithm with the pedestrian motion model and runs in near real time.
Robotics
Identifying Driver Behaviors using Trajectory Features for Vehicle Navigation
We present a novel approach to automatically identify driver behaviors from vehicle trajectories and use them for safe navigation of autonomous vehicles.
Robotics
MixedPeds: Pedestrian Detection in Unannotated Videos using Synthetically Generated Human-agents for Training
We present a new method for training pedestrian detectors on an unannotated set of images.
LCrowdV: Generating Labeled Videos for Simulation-based Crowd Behavior Learning
We present a novel procedural framework to generate an arbitrary number of labeled crowd videos (LCrowdV).
Real-time Crowd Tracking using Parameter Optimized Mixture of Motion Models
We automatically compute the optimal parameters for each of these different models based on prior tracked data and use the best model as motion prior for our particle-filter based tracking algorithm.
Realtime Multilevel Crowd Tracking using Reciprocal Velocity Obstacles
We present a novel, realtime algorithm to compute the trajectory of each pedestrian in moderately dense crowd scenes.
