Search Results for author:
Found 166 papers, 57 papers with code
DocTime: A Document-level Temporal Dependency Graph Parser
We introduce DocTime - a novel temporal dependency graph (TDG) parser that takes as input a text document and produces a temporal dependency graph.
"Don't forget to put the milk back!" Dataset for Enabling Embodied Agents to Detect Anomalous Situations
Our work assists in this goal by enabling robots to inform their users of dangerous or unsanitary anomalies in their home.
AGL-NET: Aerial-Ground Cross-Modal Global Localization with Varying Scales
We present AGL-NET, a novel learning-based method for global localization using LiDAR point clouds and satellite maps.
PoCo: Point Context Cluster for RGBD Indoor Place Recognition
We present a novel end-to-end algorithm (PoCo) for the indoor RGB-D place recognition task, aimed at identifying the most likely match for a given query frame within a reference database.
CoDa: Constrained Generation based Data Augmentation for Low-Resource NLP
We present CoDa (Constrained Generation based Data Augmentation), a controllable, effective, and training-free data augmentation technique for low-resource (data-scarce) NLP.
Do Vision-Language Models Understand Compound Nouns?
We curate Compun, a novel benchmark with 400 unique and commonly used CNs, to evaluate the effectiveness of VLMs in interpreting CNs.
Can LLMs Generate Human-Like Wayfinding Instructions? Towards Platform-Agnostic Embodied Instruction Synthesis
We present a novel approach to automatically synthesize "wayfinding instructions" for an embodied robot agent.
Global Optimality without Mixing Time Oracles in Average-reward RL via Multi-level Actor-Critic
In the context of average-reward reinforcement learning, the requirement for oracle knowledge of the mixing time, a measure of the duration a Markov chain under a fixed policy needs to achieve its stationary distribution-poses a significant challenge for the global convergence of policy gradient methods.
Right Place, Right Time! Towards ObjectNav for Non-Stationary Goals
We address this concern by inferring results on two cases for object placement: one where the objects placed follow a routine or a path, and the other where they are placed at random.
Beyond Joint Demonstrations: Personalized Expert Guidance for Efficient Multi-Agent Reinforcement Learning
Multi-Agent Reinforcement Learning (MARL) algorithms face the challenge of efficient exploration due to the exponential increase in the size of the joint state-action space.
On the Safety Concerns of Deploying LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities
However, such integration can introduce significant vulnerabilities, in terms of their susceptibility to adversarial attacks due to the language models, potentially leading to catastrophic consequences.
MaxMin-RLHF: Towards Equitable Alignment of Large Language Models with Diverse Human Preferences
Reinforcement Learning from Human Feedback (RLHF) aligns language models to human preferences by employing a singular reward model derived from preference data.
A Closer Look at the Limitations of Instruction Tuning
Our findings reveal that responses generated solely from pre-trained knowledge consistently outperform responses by models that learn any form of new knowledge from IT on open-source datasets.
REBEL: A Regularization-Based Solution for Reward Overoptimization in Robotic Reinforcement Learning from Human Feedback
Current methods to mitigate this misalignment work by learning reward functions from human preferences; however, they inadvertently introduce a risk of reward overoptimization.
Stable Distillation: Regularizing Continued Pre-training for Low-Resource Automatic Speech Recognition
Specifically, first, we perform vanilla continued pre-training on an initial SSL pre-trained model on the target domain ASR dataset and call it the teacher.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
FusDom: Combining In-Domain and Out-of-Domain Knowledge for Continuous Self-Supervised Learning
Continued pre-training (CP) offers multiple advantages, like target domain adaptation and the potential to exploit the continuous stream of unlabeled data available online.
APoLLo: Unified Adapter and Prompt Learning for Vision Language Models
Our method is designed to substantially improve the generalization capabilities of VLP models when they are fine-tuned in a few-shot setting.
AV-RIR: Audio-Visual Room Impulse Response Estimation
We propose AV-RIR, a novel multi-modal multi-task learning approach to accurately estimate the RIR from a given reverberant speech signal and the visual cues of its corresponding environment.
HawkI: Homography & Mutual Information Guidance for 3D-free Single Image to Aerial View
It seamlessly blends the visual features from the input image within a pretrained text-to-2Dimage stable diffusion model with a test-time optimization process for a careful bias-variance trade-off, which uses an Inverse Perspective Mapping (IPM) homography transformation to provide subtle cues for aerialview synthesis.
UAV-Sim: NeRF-based Synthetic Data Generation for UAV-based Perception
Using various synthetic renderers in conjunction with perception models is prevalent to create synthetic data to augment the learning in the ground-based imaging domain.
DALE: Generative Data Augmentation for Low-Resource Legal NLP
We present DALE, a novel and effective generative Data Augmentation framework for low-resource LEgal NLP.
Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey
But in parallel to the development of detection frameworks, researchers have also concentrated on designing strategies to elude detection, i. e., focusing on the impossibilities of AI-generated text detection.
HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
Our comprehensive case studies within HallusionBench shed light on the challenges of hallucination and illusion in LVLMs.
Ranked #1 on
Visual Question Answering (VQA)
on HallusionBench
Indoor Wireless Signal Modeling with Smooth Surface Diffraction Effects
We present a novel algorithm that enhances the accuracy of electromagnetic field simulations in indoor environments by incorporating the Uniform Geometrical Theory of Diffraction (UTD) for surface diffraction.
CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models
In this paper, we propose CompA, a collection of two expert-annotated benchmarks with a majority of real-world audio samples, to evaluate compositional reasoning in ALMs.
RECAP: Retrieval-Augmented Audio Captioning
We present RECAP (REtrieval-Augmented Audio CAPtioning), a novel and effective audio captioning system that generates captions conditioned on an input audio and other captions similar to the audio retrieved from a datastore.
VAPOR: Legged Robot Navigation in Outdoor Vegetation Using Offline Reinforcement Learning
We present VAPOR, a novel method for autonomous legged robot navigation in unstructured, densely vegetated outdoor environments using offline Reinforcement Learning (RL).
AdVerb: Visually Guided Audio Dereverberation
We present AdVerb, a novel audio-visual dereverberation framework that uses visual cues in addition to the reverberant sound to estimate clean audio.
ASPIRE: Language-Guided Augmentation for Robust Image Classification
This paper presents ASPIRE (Language-guided data Augmentation for SPurIous correlation REmoval), a simple yet effective solution for expanding the training dataset with synthetic images without spurious features.
PARL: A Unified Framework for Policy Alignment in Reinforcement Learning
We present a novel unified bilevel optimization-based framework, \textsf{PARL}, formulated to address the recently highlighted critical issue of policy alignment in reinforcement learning using utility or preference-based feedback.
LoLep: Single-View View Synthesis with Locally-Learned Planes and Self-Attention Occlusion Inference
We demonstrate the effectiveness of our approach and generate state-of-the-art results on different datasets.
Human Trajectory Forecasting with Explainable Behavioral Uncertainty
Model-free methods offer superior prediction accuracy but lack explainability, while model-based methods provide explainability but cannot predict well.
iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning
Our approach for intent-aware planning, iPLAN, allows agents to infer nearby drivers' intents solely from their local observations.
Ada-NAV: Adaptive Trajectory Length-Based Sample Efficient Policy Learning for Robotic Navigation
Trajectory length stands as a crucial hyperparameter within reinforcement learning (RL) algorithms, significantly contributing to the sample inefficiency in robotics applications.
ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER
In addition, we demonstrate the application of ACLM to other domains that suffer from data scarcity (e. g., biomedical).
PLAR: Prompt Learning for Action Recognition
We evaluate our approach on datasets consisting of both ground camera videos and aerial videos, and scenes with single-agent and multi-agent actions.
Ranked #1 on
Action Recognition
on Okutama-Action
BioAug: Conditional Generation based Data Augmentation for Low-Resource Biomedical NER
Though data augmentation has shown to be highly effective for low-resource NER in general, existing data augmentation techniques fail to produce factual and diverse augmentations for BioNER.
PMI Sampler: Patch Similarity Guided Frame Selection for Aerial Action Recognition
Our algorithm utilizes the motion bias within aerial videos, which enables the selection of motion-salient frames.
Ranked #1 on
Action Recognition
on UAV-Human
On the Possibilities of AI-Generated Text Detection
Our work addresses the critical issue of distinguishing text generated by Large Language Models (LLMs) from human-produced text, a task essential for numerous applications.
CrossLoc3D: Aerial-Ground Cross-Source 3D Place Recognition
We present CrossLoc3D, a novel 3D place recognition method that solves a large-scale point matching problem in a cross-source setting.
Ranked #1 on
3D Place Recognition
on CS-Campus3D
TMO: Textured Mesh Acquisition of Objects with a Mobile Device by using Differentiable Rendering
We present a new pipeline for acquiring a textured mesh in the wild with a single smartphone which offers access to images, depth maps, and valid poses.
PACE: Data-Driven Virtual Agent Interaction in Dense and Cluttered Environments
We compare our method with prior motion generating techniques and highlight the benefits of our method with a perceptual study and physical plausibility metrics.
Dynamic EM Ray Tracing for Large Urban Scenes with Multiple Receivers
We present a novel ray tracing-based radio propagation algorithm that can handle large urban scenes with hundreds or thousands of dynamic objects and receivers.
Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances
The dataset is composed of both real and synthetic videos from seven gesture classes, and is intended to support the study of synthetic-to-real domain shift for video-based action recognition.
Aerial Diffusion: Text Guided Ground-to-Aerial View Translation from a Single Image using Diffusion Models
Aerial Diffusion leverages a pretrained text-image diffusion model for prior knowledge.
RE-MOVE: An Adaptive Policy Design for Robotic Navigation Tasks in Dynamic Environments via Language-Based Feedback
Reinforcement learning-based policies for continuous control robotic navigation tasks often fail to adapt to changes in the environment during real-time deployment, which may result in catastrophic failures.
UNFUSED: UNsupervised Finetuning Using SElf supervised Distillation
Unlike prior works, which directly fine-tune a self-supervised pre-trained encoder on a target dataset, we use the encoder to generate pseudo-labels for unsupervised fine-tuning before the actual fine-tuning step.
Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation
We present LGX (Language-guided Exploration), a novel algorithm for Language-Driven Zero-Shot Object Goal Navigation (L-ZSON), where an embodied agent navigates to a uniquely described target object in a previously unseen environment.
MITFAS: Mutual Information based Temporal Feature Alignment and Sampling for Aerial Video Action Recognition
We present a novel approach for action recognition in UAV videos.
Ranked #2 on
Action Recognition
on UAV-Human
AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning
We propose a novel approach for aerial video action recognition.
Ranked #1 on
Action Recognition
on RoCoG-v2
CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network
The tremendous growth of social media users interacting in online conversations has led to significant growth in hate speech, affecting people from various demographics.
Listen2Scene: Interactive material-aware binaural sound propagation for reconstructed 3D scenes
We propose a novel neural-network-based binaural sound propagation method to generate acoustic effects for indoor 3D models of real environments.
STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning
Directed Exploration is a crucial challenge in reinforcement learning (RL), especially when rewards are sparse.
Model-based Reinforcement Learning
reinforcement-learning
+1
Beyond Exponentially Fast Mixing in Average-Reward Reinforcement Learning via Multi-Level Monte Carlo Actor-Critic
Many existing reinforcement learning (RL) methods employ stochastic gradient iteration on the back end, whose stability hinges upon a hypothesis that the data-generating process mixes exponentially fast with a rate parameter that appears in the step-size selection.
LayerDoc: Layer-wise Extraction of Spatial Hierarchical Structure in Visually-Rich Documents
Experiments show that our approach outperforms competitive baselines by 10-15% on three diverse datasets of forms and mobile app screen layouts for the tasks of spatial region classification, higher-order group identification, layout hierarchy extraction, reading order detection, and word grouping.
Synthetic Wave-Geometric Impulse Responses for Improved Speech Dereverberation
We present a novel approach to improve the performance of learning-based speech dereverberation using accurate synthetic datasets.
Towards Improved Room Impulse Response Estimation for Speech Recognition
We propose a novel approach for blind room impulse response (RIR) estimation systems in the context of a downstream application scenario, far-field automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
SLICER: Learning universal audio representations using low-resource self-supervised pre-training
We present a new Self-Supervised Learning (SSL) approach to pre-train encoders on unlabeled audio data that reduces the need for large amounts of labeled data for audio and speech classification.
MAST: Multiscale Audio Spectrogram Transformers
We present Multiscale Audio Spectrogram Transformer (MAST) for audio classification, which brings the concept of multiscale feature hierarchies to the Audio Spectrogram Transformer (AST).
MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation
Moreover, we present efficient policy distillation and tree-modification techniques that take advantage of the decision tree structure to allow improvements to a policy without retraining.
Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition
We build a differentiable static-dynamic frequency mask prior to model the salient static and dynamic pixels in the video, crucial for the underlying task of action recognition.
Placing Human Animations into 3D Scenes by Learning Interaction- and Geometry-Driven Keyframes
We compare our method, which we call PAAK, with prior approaches, including POSA, PROX ground truth, and a motion synthesis method, and highlight the benefits of our method with a perceptual study.
DC-MRTA: Decentralized Multi-Robot Task Allocation and Navigation in Complex Environments
At the higher level, we solve the task allocation by formulating it in terms of Markov Decision Processes and choosing the appropriate rewards to minimize the Total Travel Delay (TTD).
Vision-Centric BEV Perception: A Survey
In recent years, vision-centric Bird's Eye View (BEV) perception has garnered significant interest from both industry and academia due to its inherent advantages, such as providing an intuitive representation of the world and being conducive to data fusion.
A Repulsive Force Unit for Garment Collision Handling in Neural Networks
Despite recent success, deep learning-based methods for predicting 3D garment deformation under body motion suffer from interpenetration problems between the garment and the body.
Video Manipulations Beyond Faces: A Dataset with Human-Machine Analysis
To this end, we present VideoSham, a dataset consisting of 826 videos (413 real and 413 manipulated).
Human Trajectory Prediction via Neural Social Physics
Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters.
Ranked #1 on
Trajectory Prediction
on Stanford Drone
D2-TPred: Discontinuous Dependency for Trajectory Prediction under Traffic Lights
We present a trajectory prediction approach with respect to traffic lights, D2-TPred, which uses a spatial dynamic interaction graph (SDG) and a behavior dependency graph (BDG) to handle the problem of discontinuous dependency in the spatial-temporal space.
Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention
We propose a method to detect individualized highlights for users on given target videos based on their preferred highlight clips marked on previous videos they have watched.
FedBC: Calibrating Global and Local Models via Federated Learning Beyond Consensus
In this work, we quantitatively calibrate the performance of global and local models in federated learning through a multi-criterion optimization-based framework, which we cast as a constrained program.
Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies
In this paper, we present a novel Heavy-Tailed Stochastic Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems.
Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning
Model-based approaches to reinforcement learning (MBRL) exhibit favorable performance in practice, but their theoretical guarantees in large spaces are mostly restricted to the setting when transition model is Gaussian or Lipschitz, and demands a posterior estimate whose representational complexity grows unbounded with time.
SALAD: Source-free Active Label-Agnostic Domain Adaptation for Classification, Segmentation and Detection
SALAD has three key benefits: (i) it is task-agnostic, and can be applied across various visual tasks such as classification, segmentation and detection; (ii) it can handle shifts in output label space from the pre-trained source network to the target domain; (iii) it does not require access to source data for adaptation.
MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes
We show that the acoustic metrics of the IRs predicted from our MESH2IR match the ground truth with less than 10% error.
Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks
We present a learning algorithm that uses bone-driven motion networks to predict the deformation of loose-fitting garment meshes at interactive rates.
STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes
In addition, considering the property of sparse global distribution and density-varying local distribution of pedestrians, we further propose a novel method, Density-aware Hierarchical heatmap Aggregation (DHA), to enhance pedestrian perception in crowded scenes.
M-MELD: A Multilingual Multi-Party Dataset for Emotion Recognition in Conversations
Expression of emotions is a crucial part of daily human communication.
MMER: Multimodal Multi-task Learning for Speech Emotion Recognition
In this paper, we propose MMER, a novel Multimodal Multi-task learning approach for Speech Emotion Recognition.
Ranked #2 on
Speech Emotion Recognition
on IEMOCAP
(using extra training data)
3MASSIV: Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videos
We present 3MASSIV, a multilingual, multimodal and multi-aspect, expertly-annotated dataset of diverse short videos extracted from short-video social media platform - Moj.
FAR: Fourier Aerial Video Recognition
Our formulation uses a novel Fourier object disentanglement method to innately separate out the human agent (which is typically small) from the background.
Ranked #1 on
Action Recognition
on UAV Human
SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning
Given these metric poses and monocular sequences, we propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation.
Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models
Experimental results indicate that both audio and text-based models improve the emotion recognition performance and that the proposed multimodal solution achieves state-of-the-art results on the IEMOCAP benchmark.
An Intelligent Self-driving Truck System For Highway Transportation
Recently, there have been many advances in autonomous driving society, attracting a lot of attention from academia and industry.
N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks
We present a novel mesh-based learning approach (N-Cloth) for plausible 3D cloth deformation prediction.
Active Learning of Neural Collision Handler for Complex 3D Mesh Deformations
We present a robust learning algorithm to detect and handle collisions in 3D deforming meshes.
FAST-RIR: Fast neural diffuse room impulse response generator
We present a neural-network-based fast diffuse room impulse response generator (FAST-RIR) for generating room impulse responses (RIRs) for a given acoustic environment.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
HighlightMe: Detecting Highlights from Human-Centric Videos
We train our network to map the activity- and interaction-based latent structural representations of the different modalities to per-frame highlight scores based on the representativeness of the frames.
METEOR:A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviors
We present a new traffic dataset, METEOR, which captures traffic patterns and multi-agent driving behaviors in unstructured scenarios.
MotionHint: Self-Supervised Monocular Visual Odometry with Motion Constraints
A key aspect of our approach is to use an appropriate motion model that can help existing self-supervised monocular VO (SSM-VO) algorithms to overcome issues related to the local minima within their self-supervised loss functions.
DnD: Dense Depth Estimation in Crowded Dynamic Indoor Scenes
We present a novel approach for estimating depth from a monocular camera as it moves through complex and crowded indoor environments, e. g., a department store or a metro station.
Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentation
Automatic speech emotion recognition (SER) is a challenging task that plays a crucial role in natural human-computer interaction.
TIMERS: Document-level Temporal Relation Extraction
We present TIMERS - a TIME, Rhetorical and Syntactic-aware model for document-level temporal relation classification in the English language.
Ranked #3 on
Temporal Relation Classification
on TB-Dense
Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning
Our network consists of two components: a generator to synthesize gestures from a joint embedding space of features encoded from the input speech and the seed poses, and a discriminator to distinguish between the synthesized pose sequences and real 3D pose sequences.
Ranked #4 on
Gesture Generation
on TED Gesture Dataset
Improving Reverberant Speech Separation with Multi-stage Training and Curriculum Learning
We present a novel approach that improves the performance of reverberant speech separation.
M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers
We present a novel architecture for 3D object detection, M3DeTR, which combines different point cloud representations (raw, voxels, bird-eye view) with different feature scales based on multi-scale feature pyramids.
Ranked #1 on
3D Object Detection
on KITTI Cars Hard val
XAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees
In order to increase the reliability and handle the failure cases of the expert policy, we combine with a policy extraction technique to transform the resulting policy into a decision tree format.
Scene-aware Far-field Automatic Speech Recognition
We use a deep learning-based estimator to non-intrusively compute the sub-band reverberation time of an environment from its speech samples.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Robust 2D/3D Vehicle Parsing in CVIS
We combine these novel algorithms and datasets to develop a robust approach for 2D/3D vehicle parsing for CVIS.
Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality
We use an LSTM-based learning model for emotion perception.
GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments
We interface GANav with a deep reinforcement learning-based navigation algorithm and highlight its benefits in terms of navigation in real-world unstructured terrains.
Ranked #1 on
Semantic Segmentation
on RUGD
Dynamic Graph Modeling of Simultaneous EEG and Eye-tracking Data for Reading Task Identification
We present a new approach, that we call AdaGTCN, for identifying human reader intent from Electroencephalogram~(EEG) and Eye movement~(EM) data in order to help differentiate between normal reading and task-oriented reading.
Example-based Real-time Clothing Synthesis for Virtual Agents
Given a query shape and pose of the virtual agent, we synthesize the resulting clothing deformation by blending the Taylor expansion results of nearby anchoring points.
Graphics
Robust 2D/3D Vehicle Parsing in Arbitrary Camera Views for CVIS
We combine these novel algorithms and datasets to develop a robust approach for 2D/3D vehicle parsing for CVIS.
Fast 3D Acoustic Scattering via Discrete Laplacian Based Implicit Function Encoders
Acoustic properties of objects corresponding to scattering characteristics are frequently used for 3D audio content creation, environmental acoustic effects, localization and acoustic scene analysis, etc.
Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation
For autonomous driving, the dynamics and states of vehicle parts such as doors, the trunk, and the bonnet can provide meaningful semantic information and interaction states, which are essential to ensuring the safety of the self-driving vehicle.
SS-SFDA : Self-Supervised Source-Free Domain Adaptation for Road Segmentation in Hazardous Environments
We present a novel approach for unsupervised road segmentation in adverse weather conditions such as rain or fog.
Developing an Effective and Automated Patient Engagement Estimator for Telehealth: A Machine Learning Approach
We discuss MET, a learning-based algorithm proposed for perceiving a patient's level of engagement during telehealth sessions.
Sound Synthesis, Propagation, and Rendering: A Survey
To the best of our knowledge, this is the first attempt to provide a comprehensive summary of sound research in the field of computer graphics.
Sound Graphics
SelfDeco: Self-Supervised Monocular Depth Completion in Challenging Indoor Environments
We present a novel algorithm for self-supervised monocular depth completion.
B-GAP: Behavior-Rich Simulation and Navigation for Autonomous Driving
We address the problem of ego-vehicle navigation in dense simulated traffic environments populated by road agents with varying driver behaviors.
Robotics
Multi-Window Data Augmentation Approach for Speech Emotion Recognition
MWA-SER is a unimodal approach that focuses on two key concepts; designing the speech augmentation method and building the deep learning model to recognize the underlying emotion of an audio signal.
BoMuDANet: Unsupervised Adaptation for Visual Scene Understanding in Unstructured Driving Environments
We present an unsupervised adaptation approach for visual scene understanding in unstructured traffic environments.
Multi-Agent Coverage in Urban Environments
We study multi-agent coverage algorithms for autonomous monitoring and patrol in urban environments.
Robotics
PerMO: Perceiving More at Once from a Single Image for Autonomous Driving
We present a novel approach to detect, segment, and reconstruct complete textured 3D models of vehicles from a single image for autonomous driving.
AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points
Then we aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos.
MCQA: Multimodal Co-attention Based Network for Question Answering
We present MCQA, a learning-based algorithm for multimodal question answering.
OF-VO: Efficient Navigation among Pedestrians Using Commodity Sensors
Overall, our OF-VO algorithm using learning-based perception and model-based planning methods offers better performance than prior algorithms in terms of navigation time and success rate of collision avoidance.
Robotics
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues
Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake".
EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle
We report an AP of 65. 83 across 4 categories on GroupWalk, which is also an improvement over prior methods.
Ranked #2 on
Emotion Recognition in Context
on CAER
Emotion Recognition in Context
Multimodal Emotion Recognition
ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation
Our approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints.
Ranked #1 on
Emotion Classification
on EWALK
SPA: Verbal Interactions between Agents and Avatars in Shared Virtual Environments using Propositional Planning
We present a novel approach for generating plausible verbal interactions between virtual human-like agents and user avatars in shared virtual environments.
Deep Differentiable Grasp Planner for High-DOF Grippers
We present an end-to-end algorithm for training deep neural networks to grasp novel objects.
Robotics
The Liar's Walk: Detecting Deception with Gait and Gesture
We present a data-driven deep neural algorithm for detecting deceptive walking behavior using nonverbal cues like gaits and gestures.
Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization
The regularization maximizes the mutual information between navigation actions and visual observation transforms of an agent, thus promoting more informed navigation decisions.
Robotics
Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs
In practice, our approach reduces the average prediction error by more than 54% over prior algorithms and achieves a weighted average accuracy of 91. 2% for behavior prediction.
Ranked #1 on
Trajectory Prediction
on ApolloScape
Robotics
Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping
For the annotated data, we also train a classifier to map the latent embeddings to emotion labels.
Scene-Aware Audio Rendering via Deep Acoustic Analysis
We present a new method to capture the acoustic characteristics of real-world rooms using commodity devices, and use the captured characteristics to generate similar sounding sources with virtual models.
Sound Graphics Multimedia Audio and Speech Processing
M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues
Our approach combines cues from multiple co-occurring modalities (such as face, text, and speech) and also is more robust than other methods to sensor noise in any of the individual modalities.
Personality-Aware Probabilistic Map for Trajectory Prediction of Pedestrians
We update this map dynamically based on the agents in the environment and prior trajectory of a pedestrian.
STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE).
Learning Resilient Behaviors for Navigation Under Uncertainty
Deep reinforcement learning has great potential to acquire complex, adaptive behaviors for autonomous agents automatically.
DeepMNavigate: Deep Reinforced Multi-Robot Navigation Unifying Local & Global Collision Avoidance
We present a novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning (DRL).
Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding
We present a new algorithm to embed a high-dimensional configuration space of deformable objects in a low-dimensional feature space, where the configurations of objects and feature points have approximate one-to-one mapping.
Training a Constrained Natural Media Painting Agent using Reinforcement Learning
We present a novel approach to train a natural media painting using reinforcement learning.
RobustTP: End-to-End Trajectory Prediction for Heterogeneous Road-Agents in Dense Traffic with Noisy Sensor Inputs
RobustTP is an approach that first computes trajectories using a combination of a non-linear motion model and a deep learning-based instance segmentation algorithm.
Robotics
Improving Reverberant Speech Training Using Diffuse Acoustic Simulation
We present an efficient and realistic geometric acoustic simulation approach for generating and augmenting training data in speech-related machine learning tasks.
FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics
We also investigate the perception of a user in an AR setting and observe that an FVA has a statistically significant improvement in terms of the perceived friendliness and social presence of a user compared to an agent without the friendliness modeling.
RoadTrack: Realtime Tracking of Road Agents in Dense and Heterogeneous Environments
We present a realtime tracking algorithm, RoadTrack, to track heterogeneous road-agents in dense traffic videos.
Robotics
NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations
First, the latent distribution is conditioned on current observations and the target view, leading to a model-based, target-driven navigation.
LPaintB: Learning to Paint from Self-Supervision
We present a novel reinforcement learning-based natural media painting algorithm.
Identifying Emotions from Walking using Affective and Deep Features
We also present an EWalk (Emotion Walk) dataset that consists of videos of walking individuals with gaits and labeled emotions.
Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks
We present a novel learning-based approach to estimate the direction-of-arrival (DOA) of a sound source using a convolutional recurrent neural network (CRNN) trained via regression on synthetic data and Cartesian labels.
Ranked #1 on
Direction of Arrival Estimation
on SOFA
(using extra training data)
PaintBot: A Reinforcement Learning Approach for Natural Media Painting
Action selection is guided by a given reference image, which the agent attempts to replicate subject to the limitations of the action space and the agent's learned policy.
Generating Grasp Poses for a High-DOF Gripper Using Neural Networks
The quality of the grasp poses is on par with the groundtruth poses in the dataset.
Robotics
AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms
Our augmented approach combines the flexibility in a virtual environment (e. g., vehicle movements) with the richness of the real world to allow effective simulation of anywhere on earth.
TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions
We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories.
Ranked #1 on
Trajectory Prediction
on TRAF
Trajectory Prediction
Robotics
VV-Net: Voxel VAE Net with Group Convolutions for Point Cloud Segmentation
Our approach results in a good volumetric representation that effectively tackles noisy point cloud datasets and is more robust for learning.
Graphics
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).
Ranked #1 on
Trajectory Prediction
on Apolloscape Trajectory
Pedestrian Dominance Modeling for Socially-Aware Robot Navigation
We present a Pedestrian Dominance Model (PDM) to identify the dominance characteristics of pedestrians for robot navigation.
Robotics
TZC: Efficient Inter-Process Communication for Robotics Middleware with Partial Serialization
We show that by using TZC, the braking distance can be shortened by 16% than ROS.
Robotics
Data-Driven Modeling of Group Entitativity in Virtual Environments
We also present a novel interactive multi-agent simulation algorithm to model entitative groups and conduct a VR user study to validate the socio-emotional predictive power of our algorithm.
Graphics Human-Computer Interaction
Safe Navigation with Human Instructions in Complex Scenes
In this paper, we present a robotic navigation algorithm with natural language interfaces, which enables a robot to safely walk through a changing environment with moving persons by following human instructions such as "go to the restaurant and keep away from people".
Transferring Grasp Configurations using Active Learning and Local Replanning
We compute a grasp space for each part of the example object using active learning.
Robotics
PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians
Our planning system combines a POMDP algorithm with the pedestrian motion model and runs in near real time.
Robotics
Efficient Reciprocal Collision Avoidance between Heterogeneous Agents Using CTMAT
We present a novel algorithm for reciprocal collision avoidance between heterogeneous agents of different shapes and sizes.
Identifying Driver Behaviors using Trajectory Features for Vehicle Navigation
We present a novel approach to automatically identify driver behaviors from vehicle trajectories and use them for safe navigation of autonomous vehicles.
Robotics
MixedPeds: Pedestrian Detection in Unannotated Videos using Synthetically Generated Human-agents for Training
We present a new method for training pedestrian detectors on an unannotated set of images.
Efficient Generation of Motion Plans from Attribute-Based Natural Language Instructions Using Dynamic Constraint Mapping
We generate a factor graph from natural language instructions called the Dynamic Grounding Graph (DGG), which takes latent parameters into account.
Robotics
Recurrent 3D Attentional Networks for End-to-End Active Object Recognition
Active vision is inherently attention-driven: The agent actively selects views to attend in order to fast achieve the vision task while improving its internal representation of the scene being observed.
LCrowdV: Generating Labeled Videos for Simulation-based Crowd Behavior Learning
We present a novel procedural framework to generate an arbitrary number of labeled crowd videos (LCrowdV).
Exemplar-AMMs: Recognizing Crowd Movements from Pedestrian Trajectories
Specifically, we propose an optimization framework that filters out the unknown noise in the crowd trajectories and measures their similarity to the exemplar-AMMs to produce a crowd motion feature.
3D Reconstruction in the Presence of Glasses by Acoustic and Stereo Fusion
We present a novel sensor fusion algorithm that first segments the depth map into different categories such as opaque/transparent/infinity (e. g., too far to measure) and then updates the depth map based on the segmentation outcome.
Real-time Crowd Tracking using Parameter Optimized Mixture of Motion Models
We automatically compute the optimal parameters for each of these different models based on prior tracked data and use the best model as motion prior for our particle-filter based tracking algorithm.
Spoke-Darts for High-Dimensional Blue-Noise Sampling
Blue noise sampling has proved useful for many graphics applications, but remains underexplored in high-dimensional spaces due to the difficulty of generating distributions and proving properties about them.
Graphics
Realtime Multilevel Crowd Tracking using Reciprocal Velocity Obstacles
We present a novel, realtime algorithm to compute the trajectory of each pedestrian in moderately dense crowd scenes.
Leveraging Long-Term Predictions and Online-Learning in Agent-based Multiple Person Tracking
We present a multiple-person tracking algorithm, based on combining particle filters and RVO, an agent-based crowd model that infers collision-free velocities so as to predict pedestrian's motion.
