Search Results for author: Dinesh Manocha

Found 123 papers, 37 papers with code

DocTime: A Document-level Temporal Dependency Graph Parser

no code implementations NAACL 2022 Puneet Mathur, Vlad Morariu, Verena Kaynig-Fittkau, Jiuxiang Gu, Franck Dernoncourt, Quan Tran, Ani Nenkova, Dinesh Manocha, Rajiv Jain

We introduce DocTime - a novel temporal dependency graph (TDG) parser that takes as input a text document and produces a temporal dependency graph.

Dynamic EM Ray Tracing for Large Urban Scenes with Multiple Receivers

no code implementations19 Mar 2023 Ruichen Wang, Dinesh Manocha

We present a novel ray tracing-based radio propagation algorithm that can handle large urban scenes with hundreds or thousands of dynamic objects and receivers.

Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances

1 code implementation17 Mar 2023 Arun V. Reddy, Ketul Shah, William Paul, Rohita Mocharla, Judy Hoffman, Kapil D. Katyal, Dinesh Manocha, Celso M. de Melo, Rama Chellappa

The dataset is composed of both real and synthetic videos from seven gesture classes, and is intended to support the study of synthetic-to-real domain shift for video-based action recognition.

Action Recognition Domain Adaptation +1

RE-MOVE: An Adaptive Policy Design Approach for Dynamic Environments via Language-Based Feedback

no code implementations14 Mar 2023 Souradip Chakraborty, Kasun Weerakoon, Prithvi Poddar, Pratap Tokekar, Amrit Singh Bedi, Dinesh Manocha

Reinforcement learning-based policies for continuous control robotic navigation tasks often fail to adapt to changes in the environment during real-time deployment, which may result in catastrophic failures.

Continuous Control

UNFUSED: UNsupervised Finetuning Using SElf supervised Distillation

no code implementations10 Mar 2023 Ashish Seth, Sreyan Ghosh, S. Umesh, Dinesh Manocha

Unlike prior works, which directly fine-tune a self-supervised pre-trained encoder on a target dataset, we use the encoder to generate pseudo-labels for unsupervised fine-tuning before the actual fine-tuning step.

Audio Classification Self-Supervised Learning

Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Based Zero-Shot Object Navigation

no code implementations6 Mar 2023 Vishnu Sashank Dorbala, James F. Mullen Jr., Dinesh Manocha

We present LGX, a novel algorithm for Object Goal Navigation in a "language-driven, zero-shot manner", where an embodied agent navigates to an arbitrarily described target object in a previously unexplored environment.

Motion Planning object-detection +2

CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network

no code implementations2 Mar 2023 Sreyan Ghosh, Manan Suri, Purva Chiniya, Utkarsh Tyagi, Sonal Kumar, Dinesh Manocha

The tremendous growth of social media users interacting in online conversations has also led to significant growth in hate speech.

Scene2BIR: Material-aware learning-based binaural impulse response generator for reconstructed real-world 3D scenes

no code implementations2 Feb 2023 Anton Jeran Ratnarajah, Dinesh Manocha

We present an end-to-end binaural impulse response generator (BIR) to generate plausible sounds in real-time for real-world models.

Beyond Exponentially Fast Mixing in Average-Reward Reinforcement Learning via Multi-Level Monte Carlo Actor-Critic

no code implementations28 Jan 2023 Wesley A. Suttle, Amrit Singh Bedi, Bhrij Patel, Brian M. Sadler, Alec Koppel, Dinesh Manocha

Many existing reinforcement learning (RL) methods employ stochastic gradient iteration on the back end, whose stability hinges upon a hypothesis that the data-generating process mixes exponentially fast with a rate parameter that appears in the step-size selection.

Synthetic Wave-Geometric Impulse Responses for Improved Speech Dereverberation

no code implementations10 Dec 2022 Rohith Aralikatti, Zhenyu Tang, Dinesh Manocha

We present a novel approach to improve the performance of learning-based speech dereverberation using accurate synthetic datasets.

Speech Dereverberation

Towards Improved Room Impulse Response Estimation for Speech Recognition

no code implementations8 Nov 2022 Anton Ratnarajah, Ishwarya Ananthabhotla, Vamsi Krishna Ithapu, Pablo Hoffmann, Dinesh Manocha, Paul Calamia

We propose a novel approach for blind room impulse response (RIR) estimation systems in the context of a downstream application scenario, far-field automatic speech recognition (ASR).

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

MAST: Multiscale Audio Spectrogram Transformers

no code implementations2 Nov 2022 Sreyan Ghosh, Ashish Seth, S. Umesh, Dinesh Manocha

We present Multiscale Audio Spectrogram Transformer (MAST) for audio classification, which brings the concept of multiscale feature hierarchies to the Audio Spectrogram Transformer (AST).

Audio Classification Self-Supervised Learning

SLICER: Learning universal audio representations using low-resource self-supervised pre-training

no code implementations2 Nov 2022 Ashish Seth, Sreyan Ghosh, S. Umesh, Dinesh Manocha

We present a new Self-Supervised Learning (SSL) approach to pre-train encoders on unlabeled audio data that reduces the need for large amounts of labeled data for audio and speech classification.

Contrastive Learning Representation Learning +1

MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation

no code implementations19 Sep 2022 Aaron M. Roth, Jing Liang, Ram Sriram, Elham Tabassi, Dinesh Manocha

Moreover, we present efficient policy distillation and tree-modification techniques that take advantage of the decision tree structure to allow improvements to a policy without retraining.

Imitation Learning reinforcement-learning +2

Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition

no code implementations15 Sep 2022 Divya Kothandaraman, Ming Lin, Dinesh Manocha

We build a differentiable static-dynamic frequency mask prior to model the salient static and dynamic pixels in the video, crucial for the underlying task of action recognition.

Action Recognition Activity Recognition In Videos +2

Placing Human Animations into 3D Scenes by Learning Interaction- and Geometry-Driven Keyframes

no code implementations13 Sep 2022 James F. Mullen Jr, Divya Kothandaraman, Aniket Bera, Dinesh Manocha

We compare our method, which we call PAAK, with prior approaches, including POSA, PROX ground truth, and a motion synthesis method, and highlight the benefits of our method with a perceptual study.

Motion Synthesis

DC-MRTA: Decentralized Multi-Robot Task Allocation and Navigation in Complex Environments

no code implementations7 Sep 2022 Aakriti Agrawal, Senthil Hariharan, Amrit Singh Bedi, Dinesh Manocha

At the higher level, we solve the task allocation by formulating it in terms of Markov Decision Processes and choosing the appropriate rewards to minimize the Total Travel Delay (TTD).

Vision-Centric BEV Perception: A Survey

1 code implementation4 Aug 2022 Yuexin Ma, Tai Wang, Xuyang Bai, Huitong Yang, Yuenan Hou, Yaming Wang, Yu Qiao, Ruigang Yang, Dinesh Manocha, Xinge Zhu

To stimulate its future research, this paper presents a comprehensive survey of recent progress of vision-centric BEV perception and its extensions.

A Repulsive Force Unit for Garment Collision Handling in Neural Networks

no code implementations28 Jul 2022 Qingyang Tan, Yi Zhou, Tuanfeng Wang, Duygu Ceylan, Xin Sun, Dinesh Manocha

Despite recent success, deep learning-based methods for predicting 3D garment deformation under body motion suffer from interpenetration problems between the garment and the body.

Human Trajectory Prediction via Neural Social Physics

1 code implementation21 Jul 2022 Jiangbei Yue, Dinesh Manocha, He Wang

Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters.

Inductive Bias Trajectory Prediction

D2-TPred: Discontinuous Dependency for Trajectory Prediction under Traffic Lights

1 code implementation21 Jul 2022 Yuzhen Zhang, Wentong Wang, Weizhi Guo, Pei Lv, Mingliang Xu, Wei Chen, Dinesh Manocha

We present a trajectory prediction approach with respect to traffic lights, D2-TPred, which uses a spatial dynamic interaction graph (SDG) and a behavior dependency graph (BDG) to handle the problem of discontinuous dependency in the spatial-temporal space.

Trajectory Prediction

Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention

no code implementations18 Jul 2022 Uttaran Bhattacharya, Gang Wu, Stefano Petrangeli, Viswanathan Swaminathan, Dinesh Manocha

We propose a method to detect individualized highlights for users on given target videos based on their preferred highlight clips marked on previous videos they have watched.

Highlight Detection

FedBC: Calibrating Global and Local Models via Federated Learning Beyond Consensus

no code implementations22 Jun 2022 Amrit Singh Bedi, Chen Fan, Alec Koppel, Anit Kumar Sahu, Brian M. Sadler, Furong Huang, Dinesh Manocha

In this work, we quantitatively calibrate the performance of global and local models in federated learning through a multi-criterion optimization-based framework, which we cast as a constrained program.

Federated Learning

Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies

no code implementations12 Jun 2022 Souradip Chakraborty, Amrit Singh Bedi, Alec Koppel, Pratap Tokekar, Dinesh Manocha

In this paper, we present a novel Heavy-Tailed Stochastic Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems.

Continuous Control OpenAI Gym

Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning

no code implementations2 Jun 2022 Souradip Chakraborty, Amrit Singh Bedi, Alec Koppel, Brian M. Sadler, Furong Huang, Pratap Tokekar, Dinesh Manocha

In this work, we propose a novel ${\bf K}$ernelized ${\bf S}$tein Discrepancy-based Posterior Sampling for ${\bf RL}$ algorithm (named $\texttt{KSRL}$) which extends model-based RL based upon posterior sampling (PSRL) in several ways: we (i) relax the need for any smoothness or Gaussian assumptions, allowing for complex mixture models; (ii) ensure it is applicable to large-scale training by incorporating a compression step such that the posterior consists of a \emph{Bayesian coreset} of only statistically significant past state-action pairs; and (iii) develop a novel regret analysis of PSRL based upon integral probability metrics, which, under a smoothness condition on the constructed posterior, can be evaluated in closed form as the kernelized Stein discrepancy (KSD).

Continuous Control Model-based Reinforcement Learning +2

SALAD: Source-free Active Label-Agnostic Domain Adaptation for Classification, Segmentation and Detection

1 code implementation24 May 2022 Divya Kothandaraman, Sumit Shekhar, Abhilasha Sancheti, Manoj Ghuhan, Tripti Shukla, Dinesh Manocha

SALAD has three key benefits: (i) it is task-agnostic, and can be applied across various visual tasks such as classification, segmentation and detection; (ii) it can handle shifts in output label space from the pre-trained source network to the target domain; (iii) it does not require access to source data for adaptation.

Active Learning Domain Adaptation +2

Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks

1 code implementation3 May 2022 Xiaoyu Pan, Jiaming Mai, Xinwei Jiang, Dongxue Tang, Jingxiang Li, Tianjia Shao, Kun Zhou, Xiaogang Jin, Dinesh Manocha

We present a learning algorithm that uses bone-driven motion networks to predict the deformation of loose-fitting garment meshes at interactive rates.

STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes

1 code implementation CVPR 2022 Peishan Cong, Xinge Zhu, Feng Qiao, Yiming Ren, Xidong Peng, Yuenan Hou, Lan Xu, Ruigang Yang, Dinesh Manocha, Yuexin Ma

In addition, considering the property of sparse global distribution and density-varying local distribution of pedestrians, we further propose a novel method, Density-aware Hierarchical heatmap Aggregation (DHA), to enhance pedestrian perception in crowded scenes.

Pedestrian Detection

3MASSIV: Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videos

no code implementations CVPR 2022 Vikram Gupta, Trisha Mittal, Puneet Mathur, Vaibhav Mishra, Mayank Maheshwari, Aniket Bera, Debdoot Mukherjee, Dinesh Manocha

We present 3MASSIV, a multilingual, multimodal and multi-aspect, expertly-annotated dataset of diverse short videos extracted from short-video social media platform - Moj.

FAR: Fourier Aerial Video Recognition

1 code implementation21 Mar 2022 Divya Kothandaraman, Tianrui Guan, Xijun Wang, Sean Hu, Ming Lin, Dinesh Manocha

Our formulation uses a novel Fourier object disentanglement method to innately separate out the human agent (which is typically small) from the background.

Activity Recognition Disentanglement +1

SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning

no code implementations10 Mar 2022 Jaehoon Choi, Dongki Jung, Yonghan Lee, Deokhwa Kim, Dinesh Manocha, Donghwan Lee

Given these metric poses and monocular sequences, we propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation.

Monocular Depth Estimation Robot Navigation +2

Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models

no code implementations16 Feb 2022 Sarala Padi, Seyed Omid Sadjadi, Dinesh Manocha, Ram D. Sriram

Experimental results indicate that both audio and text-based models improve the emotion recognition performance and that the proposed multimodal solution achieves state-of-the-art results on the IEMOCAP benchmark.

Data Augmentation Emotional Intelligence +3

N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks

no code implementations13 Dec 2021 Yudi Li, Min Tang, Yun Yang, Zi Huang, Ruofeng Tong, Shuangcai Yang, Yao Li, Dinesh Manocha

We present a novel mesh-based learning approach (N-Cloth) for plausible 3D cloth deformation prediction.

FAST-RIR: Fast neural diffuse room impulse response generator

2 code implementations7 Oct 2021 Anton Ratnarajah, Shi-Xiong Zhang, Meng Yu, Zhenyu Tang, Dinesh Manocha, Dong Yu

We present a neural-network-based fast diffuse room impulse response generator (FAST-RIR) for generating room impulse responses (RIRs) for a given acoustic environment.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

HighlightMe: Detecting Highlights from Human-Centric Videos

no code implementations ICCV 2021 Uttaran Bhattacharya, Gang Wu, Stefano Petrangeli, Viswanathan Swaminathan, Dinesh Manocha

We train our network to map the activity- and interaction-based latent structural representations of the different modalities to per-frame highlight scores based on the representativeness of the frames.

MotionHint: Self-Supervised Monocular Visual Odometry with Motion Constraints

1 code implementation14 Sep 2021 Cong Wang, Yu-Ping Wang, Dinesh Manocha

A key aspect of our approach is to use an appropriate motion model that can help existing self-supervised monocular VO (SSM-VO) algorithms to overcome issues related to the local minima within their self-supervised loss functions.

Monocular Visual Odometry

DnD: Dense Depth Estimation in Crowded Dynamic Indoor Scenes

no code implementations ICCV 2021 Dongki Jung, Jaehoon Choi, Yonghan Lee, Deokhwa Kim, Changick Kim, Dinesh Manocha, Donghwan Lee

We present a novel approach for estimating depth from a monocular camera as it moves through complex and crowded indoor environments, e. g., a department store or a metro station.

3D Reconstruction Depth Estimation

Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentation

no code implementations5 Aug 2021 Sarala Padi, Seyed Omid Sadjadi, Dinesh Manocha, Ram D. Sriram

Automatic speech emotion recognition (SER) is a challenging task that plays a crucial role in natural human-computer interaction.

Emotion Classification Speaker Recognition +2

Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning

1 code implementation31 Jul 2021 Uttaran Bhattacharya, Elizabeth Childs, Nicholas Rewkowski, Dinesh Manocha

Our network consists of two components: a generator to synthesize gestures from a joint embedding space of features encoded from the input speech and the seed poses, and a discriminator to distinguish between the synthesized pose sequences and real 3D pose sequences.

Gesture Generation

M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers

1 code implementation24 Apr 2021 Tianrui Guan, Jun Wang, Shiyi Lan, Rohan Chandra, Zuxuan Wu, Larry Davis, Dinesh Manocha

We present a novel architecture for 3D object detection, M3DeTR, which combines different point cloud representations (raw, voxels, bird-eye view) with different feature scales based on multi-scale feature pyramids.

3D Object Detection object-detection

XAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees

1 code implementation22 Apr 2021 Aaron M. Roth, Jing Liang, Dinesh Manocha

In order to increase the reliability and handle the failure cases of the expert policy, we combine with a policy extraction technique to transform the resulting policy into a decision tree format.

Robot Navigation

Scene-aware Far-field Automatic Speech Recognition

no code implementations21 Apr 2021 Zhenyu Tang, Dinesh Manocha

We use a deep learning-based estimator to non-intrusively compute the sub-band reverberation time of an environment from its speech samples.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Robust 2D/3D Vehicle Parsing in CVIS

no code implementations11 Mar 2021 Hui Miao, Feixiang Lu, Zongdai Liu, Liangjun Zhang, Dinesh Manocha, Bin Zhou

We combine these novel algorithms and datasets to develop a robust approach for 2D/3D vehicle parsing for CVIS.

Benchmarking Data Augmentation +4

GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments

1 code implementation7 Mar 2021 Tianrui Guan, Divya Kothandaraman, Rohan Chandra, Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Dinesh Manocha

We interface GANav with a deep reinforcement learning-based navigation algorithm and highlight its benefits in terms of navigation in real-world unstructured terrains.

Robot Navigation Semantic Segmentation

Dynamic Graph Modeling of Simultaneous EEG and Eye-tracking Data for Reading Task Identification

no code implementations21 Feb 2021 Puneet Mathur, Trisha Mittal, Dinesh Manocha

We present a new approach, that we call AdaGTCN, for identifying human reader intent from Electroencephalogram~(EEG) and Eye movement~(EM) data in order to help differentiate between normal reading and task-oriented reading.

Electroencephalogram (EEG) Graph Learning

Example-based Real-time Clothing Synthesis for Virtual Agents

no code implementations8 Jan 2021 Nannan Wu, Qianwen Chao, Yanzhen Chen, Weiwei Xu, Chen Liu, Dinesh Manocha, Wenxin Sun, Yi Han, Xinran Yao, Xiaogang Jin

Given a query shape and pose of the virtual agent, we synthesize the resulting clothing deformation by blending the Taylor expansion results of nearby anchoring points.


Fast 3D Acoustic Scattering via Discrete Laplacian Based Implicit Function Encoders

no code implementations1 Jan 2021 Hsien-Yu Meng, Zhenyu Tang, Dinesh Manocha

Acoustic properties of objects corresponding to scattering characteristics are frequently used for 3D audio content creation, environmental acoustic effects, localization and acoustic scene analysis, etc.

Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation

1 code implementation15 Dec 2020 Feixiang Lu, Zongdai Liu, Hui Miao, Peng Wang, Liangjun Zhang, Ruigang Yang, Dinesh Manocha, Bin Zhou

For autonomous driving, the dynamics and states of vehicle parts such as doors, the trunk, and the bonnet can provide meaningful semantic information and interaction states, which are essential to ensuring the safety of the self-driving vehicle.

Autonomous Driving Data Augmentation +3

Sound Synthesis, Propagation, and Rendering: A Survey

no code implementations11 Nov 2020 Shiguang Liu, Dinesh Manocha

To the best of our knowledge, this is the first attempt to provide a comprehensive summary of sound research in the field of computer graphics.

Sound Graphics

B-GAP: Behavior-Rich Simulation and Navigation for Autonomous Driving

3 code implementations7 Nov 2020 Angelos Mavrogiannis, Rohan Chandra, Dinesh Manocha

We address the problem of ego-vehicle navigation in dense simulated traffic environments populated by road agents with varying driver behaviors.


Multi-Window Data Augmentation Approach for Speech Emotion Recognition

no code implementations19 Oct 2020 Sarala Padi, Dinesh Manocha, Ram D. Sriram

MWA-SER is a unimodal approach that focuses on two key concepts; designing the speech augmentation method and building the deep learning model to recognize the underlying emotion of an audio signal.

Data Augmentation Speech Emotion Recognition

Multi-Agent Coverage in Urban Environments

1 code implementation17 Aug 2020 Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin, Dinesh Manocha, Huan Xu, Michael Otte

We study multi-agent coverage algorithms for autonomous monitoring and patrol in urban environments.


PerMO: Perceiving More at Once from a Single Image for Autonomous Driving

no code implementations16 Jul 2020 Feixiang Lu, Zongdai Liu, Xibin Song, Dingfu Zhou, Wei Li, Hui Miao, Miao Liao, Liangjun Zhang, Bin Zhou, Ruigang Yang, Dinesh Manocha

We present a novel approach to detect, segment, and reconstruct complete textured 3D models of vehicles from a single image for autonomous driving.

3D Reconstruction Autonomous Driving +3

OF-VO: Efficient Navigation among Pedestrians Using Commodity Sensors

no code implementations23 Apr 2020 Jing Liang, Yi-Ling Qiao, Dinesh Manocha

Overall, our OF-VO algorithm using learning-based perception and model-based planning methods offers better performance than prior algorithms in terms of navigation time and success rate of collision avoidance.


Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues

no code implementations14 Mar 2020 Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, Dinesh Manocha

Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake".

DeepFake Detection Face Swapping

ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation

1 code implementation2 Mar 2020 Venkatraman Narayanan, Bala Murali Manoghar, Vishnu Sashank Dorbala, Dinesh Manocha, Aniket Bera

Our approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints.

Emotion Classification Emotion Recognition +3

SPA: Verbal Interactions between Agents and Avatars in Shared Virtual Environments using Propositional Planning

no code implementations8 Feb 2020 Andrew Best, Sahil Narang, Dinesh Manocha

We present a novel approach for generating plausible verbal interactions between virtual human-like agents and user avatars in shared virtual environments.

Deep Differentiable Grasp Planner for High-DOF Grippers

no code implementations4 Feb 2020 Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha

We present an end-to-end algorithm for training deep neural networks to grasp novel objects.


The Liar's Walk: Detecting Deception with Gait and Gesture

no code implementations14 Dec 2019 Tanmay Randhavane, Uttaran Bhattacharya, Kyra Kapsaskis, Kurt Gray, Aniket Bera, Dinesh Manocha

We present a data-driven deep neural algorithm for detecting deceptive walking behavior using nonverbal cues like gaits and gestures.

Action Classification

Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization

1 code implementation9 Dec 2019 Qiaoyun Wu, Kai Xu, Jun Wang, Mingliang Xu, Dinesh Manocha

The regularization maximizes the mutual information between navigation actions and visual observation transforms of an agent, thus promoting more informed navigation decisions.


Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs

no code implementations arXiv 2019 Rohan Chandra, Tianrui Guan, Srujan Panuganti, Trisha Mittal, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

In practice, our approach reduces the average prediction error by more than 54% over prior algorithms and achieves a weighted average accuracy of 91. 2% for behavior prediction.


Scene-Aware Audio Rendering via Deep Acoustic Analysis

no code implementations14 Nov 2019 Zhenyu Tang, Nicholas J. Bryan, DIngzeyu Li, Timothy R. Langlois, Dinesh Manocha

We present a new method to capture the acoustic characteristics of real-world rooms using commodity devices, and use the captured characteristics to generate similar sounding sources with virtual models.

Sound Graphics Multimedia Audio and Speech Processing

M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues

no code implementations9 Nov 2019 Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, Dinesh Manocha

Our approach combines cues from multiple co-occurring modalities (such as face, text, and speech) and also is more robust than other methods to sensor noise in any of the individual modalities.

Multimodal Emotion Recognition

Personality-Aware Probabilistic Map for Trajectory Prediction of Pedestrians

no code implementations1 Nov 2019 Chaochao Li, Pei Lv, Mingliang Xu, Xinyu Wang, Dinesh Manocha, Bing Zhou, Meng Wang

We update this map dynamically based on the agents in the environment and prior trajectory of a pedestrian.

Trajectory Prediction

STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits

1 code implementation28 Oct 2019 Uttaran Bhattacharya, Trisha Mittal, Rohan Chandra, Tanmay Randhavane, Aniket Bera, Dinesh Manocha

We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE).

General Classification

Learning Resilient Behaviors for Navigation Under Uncertainty

no code implementations22 Oct 2019 Tingxiang Fan, Pinxin Long, Wenxi Liu, Jia Pan, Ruigang Yang, Dinesh Manocha

Deep reinforcement learning has great potential to acquire complex, adaptive behaviors for autonomous agents automatically.

Autonomous Driving

DeepMNavigate: Deep Reinforced Multi-Robot Navigation Unifying Local & Global Collision Avoidance

no code implementations4 Oct 2019 Qingyang Tan, Tingxiang Fan, Jia Pan, Dinesh Manocha

We present a novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning (DRL).

reinforcement-learning Reinforcement Learning (RL) +1

Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding

no code implementations26 Sep 2019 Qingyang Tan, Zherong Pan, Lin Gao, Dinesh Manocha

We present a new algorithm to embed a high-dimensional configuration space of deformable objects in a low-dimensional feature space, where the configurations of objects and feature points have approximate one-to-one mapping.

Dimensionality Reduction Robot Manipulation

RobustTP: End-to-End Trajectory Prediction for Heterogeneous Road-Agents in Dense Traffic with Noisy Sensor Inputs

1 code implementation20 Jul 2019 Rohan Chandra, Uttaran Bhattacharya, Christian Roncal, Aniket Bera, Dinesh Manocha

RobustTP is an approach that first computes trajectories using a combination of a non-linear motion model and a deep learning-based instance segmentation algorithm.


Improving Reverberant Speech Training Using Diffuse Acoustic Simulation

no code implementations9 Jul 2019 Zhenyu Tang, Lian-Wu Chen, Bo Wu, Dong Yu, Dinesh Manocha

We present an efficient and realistic geometric acoustic simulation approach for generating and augmenting training data in speech-related machine learning tasks.

BIG-bench Machine Learning Keyword Spotting +2

FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics

no code implementations30 Jun 2019 Tanmay Randhavane, Aniket Bera, Kyra Kapsaskis, Kurt Gray, Dinesh Manocha

We also investigate the perception of a user in an AR setting and observe that an FVA has a statistically significant improvement in terms of the perceived friendliness and social presence of a user compared to an agent without the friendliness modeling.

RoadTrack: Realtime Tracking of Road Agents in Dense and Heterogeneous Environments

1 code implementation25 Jun 2019 Rohan Chandra, Uttaran Bhattacharya, Tanmay Randhavane, Aniket Bera, Dinesh Manocha

We present a realtime tracking algorithm, RoadTrack, to track heterogeneous road-agents in dense traffic videos.


NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations

1 code implementation17 Jun 2019 Qiaoyun Wu, Dinesh Manocha, Jun Wang, Kai Xu

First, the latent distribution is conditioned on current observations and the target view, leading to a model-based, target-driven navigation.

Visual Navigation

Identifying Emotions from Walking using Affective and Deep Features

no code implementations14 Jun 2019 Tanmay Randhavane, Uttaran Bhattacharya, Kyra Kapsaskis, Kurt Gray, Aniket Bera, Dinesh Manocha

We also present an EWalk (Emotion Walk) dataset that consists of videos of walking individuals with gaits and labeled emotions.

Emotion Recognition

Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks

1 code implementation17 Apr 2019 Zhenyu Tang, John D. Kanu, Kevin Hogan, Dinesh Manocha

We present a novel learning-based approach to estimate the direction-of-arrival (DOA) of a sound source using a convolutional recurrent neural network (CRNN) trained via regression on synthetic data and Cartesian labels.

 Ranked #1 on Direction of Arrival Estimation on SOFA (using extra training data)

Direction of Arrival Estimation General Classification +1

PaintBot: A Reinforcement Learning Approach for Natural Media Painting

no code implementations3 Apr 2019 Biao Jia, Chen Fang, Jonathan Brandt, Byungmoon Kim, Dinesh Manocha

Action selection is guided by a given reference image, which the agent attempts to replicate subject to the limitations of the action space and the agent's learned policy.

reinforcement-learning Reinforcement Learning (RL)

Generating Grasp Poses for a High-DOF Gripper Using Neural Networks

no code implementations1 Mar 2019 Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha

The quality of the grasp poses is on par with the groundtruth poses in the dataset.


AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms

1 code implementation23 Jan 2019 Wei Li, Chengwei Pan, Rong Zhang, Jiaping Ren, Yuexin Ma, Jin Fang, Feilong Yan, Qichuan Geng, Xinyu Huang, Huajun Gong, Weiwei Xu, Guoping Wang, Dinesh Manocha, Ruigang Yang

Our augmented approach combines the flexibility in a virtual environment (e. g., vehicle movements) with the richness of the real world to allow effective simulation of anywhere on earth.

Autonomous Driving

TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions

2 code implementations CVPR 2019 Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories.

 Ranked #1 on Trajectory Prediction on NGSIM (RMSE metric)


VV-Net: Voxel VAE Net with Group Convolutions for Point Cloud Segmentation

1 code implementation 2019 IEEE/CVF International Conference on Computer Vision (ICCV) 2020 Hsien-Yu Meng, Lin Gao, Yu-Kun Lai, Dinesh Manocha

Our approach results in a good volumetric representation that effectively tackles noisy point cloud datasets and is more robust for learning.


TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents

1 code implementation6 Nov 2018 Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, Dinesh Manocha

To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).

Autonomous Vehicles Navigate +2

Pedestrian Dominance Modeling for Socially-Aware Robot Navigation

no code implementations15 Oct 2018 Tanmay Randhavane, Aniket Bera, Emily Kubin, Austin Wang, Kurt Gray, Dinesh Manocha

We present a Pedestrian Dominance Model (PDM) to identify the dominance characteristics of pedestrians for robot navigation.


Data-Driven Modeling of Group Entitativity in Virtual Environments

no code implementations28 Sep 2018 Aniket Bera, Tanmay Randhavane, Emily Kubin, Husam Shaik, Kurt Gray, Dinesh Manocha

We also present a novel interactive multi-agent simulation algorithm to model entitative groups and conduct a VR user study to validate the socio-emotional predictive power of our algorithm.

Graphics Human-Computer Interaction

Safe Navigation with Human Instructions in Complex Scenes

no code implementations12 Sep 2018 Zhe Hu, Jia Pan, Tingxiang Fan, Ruigang Yang, Dinesh Manocha

In this paper, we present a robotic navigation algorithm with natural language interfaces, which enables a robot to safely walk through a changing environment with moving persons by following human instructions such as "go to the restaurant and keep away from people".

Motion Planning object-detection +1

Transferring Grasp Configurations using Active Learning and Local Replanning

no code implementations22 Jul 2018 Hao Tian, Changbo Wang, Dinesh Manocha, Xin-Yu Zhang

We compute a grasp space for each part of the example object using active learning.


PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians

no code implementations30 May 2018 Yuanfu Luo, Panpan Cai, Aniket Bera, David Hsu, Wee Sun Lee, Dinesh Manocha

Our planning system combines a POMDP algorithm with the pedestrian motion model and runs in near real time.


Efficient Reciprocal Collision Avoidance between Heterogeneous Agents Using CTMAT

no code implementations7 Apr 2018 Yuexin Ma, Dinesh Manocha, Wenping Wang

We present a novel algorithm for reciprocal collision avoidance between heterogeneous agents of different shapes and sizes.

Identifying Driver Behaviors using Trajectory Features for Vehicle Navigation

no code implementations2 Mar 2018 Ernest Cheung, Aniket Bera, Emily Kubin, Kurt Gray, Dinesh Manocha

We present a novel approach to automatically identify driver behaviors from vehicle trajectories and use them for safe navigation of autonomous vehicles.


Efficient Generation of Motion Plans from Attribute-Based Natural Language Instructions Using Dynamic Constraint Mapping

no code implementations8 Jul 2017 Jae Sung Park, Biao Jia, Mohit Bansal, Dinesh Manocha

We generate a factor graph from natural language instructions called the Dynamic Grounding Graph (DGG), which takes latent parameters into account.


Recurrent 3D Attentional Networks for End-to-End Active Object Recognition

no code implementations14 Oct 2016 Min Liu, Yifei Shi, Lintao Zheng, Kai Xu, Hui Huang, Dinesh Manocha

Active vision is inherently attention-driven: The agent actively selects views to attend in order to fast achieve the vision task while improving its internal representation of the scene being observed.

Object Recognition

Exemplar-AMMs: Recognizing Crowd Movements from Pedestrian Trajectories

no code implementations31 Mar 2016 Wenxi Liu, Rynson W. H. Lau, Xiaogang Wang, Dinesh Manocha

Specifically, we propose an optimization framework that filters out the unknown noise in the crowd trajectories and measures their similarity to the exemplar-AMMs to produce a crowd motion feature.

Multi-Label Classification

3D Reconstruction in the Presence of Glasses by Acoustic and Stereo Fusion

no code implementations CVPR 2015 Mao Ye, Yu Zhang, Ruigang Yang, Dinesh Manocha

We present a novel sensor fusion algorithm that first segments the depth map into different categories such as opaque/transparent/infinity (e. g., too far to measure) and then updates the depth map based on the segmentation outcome.

3D Reconstruction Transparent objects

Real-time Crowd Tracking using Parameter Optimized Mixture of Motion Models

no code implementations16 Sep 2014 Aniket Bera, David Wolinski, Julien Pettré, Dinesh Manocha

We automatically compute the optimal parameters for each of these different models based on prior tracked data and use the best model as motion prior for our particle-filter based tracking algorithm.

Combinatorial Optimization

Spoke-Darts for High-Dimensional Blue-Noise Sampling

1 code implementation5 Aug 2014 Scott A. Mitchell, Mohamed S. Ebeida, Muhammad A. Awad, Chonhyon Park, Anjul Patney, Ahmad A. Rushdi, Laura P. Swiler, Dinesh Manocha, Li-Yi Wei

Blue noise sampling has proved useful for many graphics applications, but remains underexplored in high-dimensional spaces due to the difficulty of generating distributions and proving properties about them.


Realtime Multilevel Crowd Tracking using Reciprocal Velocity Obstacles

no code implementations11 Feb 2014 Aniket Bera, Dinesh Manocha

We present a novel, realtime algorithm to compute the trajectory of each pedestrian in moderately dense crowd scenes.

Leveraging Long-Term Predictions and Online-Learning in Agent-based Multiple Person Tracking

no code implementations10 Feb 2014 Wenxi Liu, Antoni B. Chan, Rynson W. H. Lau, Dinesh Manocha

We present a multiple-person tracking algorithm, based on combining particle filters and RVO, an agent-based crowd model that infers collision-free velocities so as to predict pedestrian's motion.

Cannot find the paper you are looking for? You can Submit a new open access paper.