Search Results for author:
Found 57 papers, 14 papers with code
A Linear Time and Space Local Point Cloud Geometry Encoder via Vectorized Kernel Mixture (VecKM)
We propose VecKM, a novel local point cloud geometry encoder that is descriptive, efficient and robust to noise.
TimeRewind: Rewinding Time with Image-and-Events Video Diffusion
Through extensive experimentation, we demonstrate the capability of our approach to synthesize high-quality videos that effectively ``rewind'' time, showcasing the potential of combining event camera technology with generative models.
MARVIS: Motion & Geometry Aware Real and Virtual Image Segmentation
By creating realistic synthetic images that mimic the complexities of the water surface, we provide fine-grained training data for our network (MARVIS) to discern between real and virtual images effectively.
NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction
NatSGD serves as a foundational resource at the intersection of machine learning and HRI research, and we demonstrate its effectiveness in training robots to understand tasks through multimodal human commands, emphasizing the significance of jointly considering speech and gestures.
LEAP: LLM-Generation of Egocentric Action Programs
We apply LEAP over a majority (87\%) of the training set of the EPIC Kitchens dataset, and release the resulting action programs as a publicly available dataset here (https://drive. google. com/drive/folders/1Cpkw_TI1IIxXdzor0pOXG3rWJWuKU5Ex? usp=drive_link).
Decodable and Sample Invariant Continuous Object Encoder
In addition, the encoding is decodable, which enables neural networks to regress continuous objects by regressing their encodings.
AcTExplore: Active Tactile Exploration on Unknown Objects
Tactile exploration plays a crucial role in understanding object structures for fundamental robotics tasks such as grasping and manipulation.
Detecting Olives with Synthetic or Real Data? Olive the Above
When access to real, human-labeled data is limited, a combination of mostly synthetic data and a small amount of real data can enhance olive detection.
Whale Detection Enhancement through Synthetic Satellite Images
With a number of marine populations in rapid decline, collecting and analyzing data about marine populations has become increasingly important to develop effective conservation policies for a wide range of marine animals, including whales.
Therbligs in Action: Video Understanding through Motion Primitives
In this paper we introduce a rule-based, compositional, and hierarchical modeling of action using Therbligs as our atoms.
WorldGen: A Large Scale Generative Simulator
In the era of deep learning, data is the critical determining factor in the performance of neural network models.
OysterNet: Enhanced Oyster Detection Using Simulation
However, systems for accurate oyster detection require large datasets obtaining which is an expensive and labor-intensive task in underwater environments.
Gluing Neural Networks Symbolically Through Hyperdimensional Computing
This process can be performed iteratively and even on single neural networks by instead making a consensus of multiple classification hypervectors.
EVIMO2: An Event Camera Dataset for Motion Segmentation, Optical Flow, Structure from Motion, and Visual Inertial Odometry in Indoor Scenes with Monocular or Stereo Algorithms
Depth and segmentation are provided at 60 Hz for the event cameras and 30 Hz for the classical camera.
AIMusicGuru: Music Assisted Human Pose Correction
Pose Estimation techniques rely on visual cues available through observations represented in the form of pixels.
DiffPoseNet: Direct Differentiable Camera Pose Estimation
In this paper, we introduce a network NFlowNet, for normal flow estimation which is used to enforce robust and direct constraints.
TTCDist: Fast Distance Estimation From an Active Monocular Camera Using Time-to-Contact
Distance estimation from vision is fundamental for a myriad of robotic applications such as navigation, manipulation, and planning.
NudgeSeg: Zero-Shot Object Segmentation by Repeated Physical Interaction
In this paper, we present the first framework to segment unknown objects in a cluttered scene by repeatedly 'nudging' at the objects and moving them to obtain additional motion cues at every step using only a monochrome monocular camera.
EVPropNet: Detecting Drones By Finding Propellers For Mid-Air Landing And Following
Our network can detect propellers at a rate of 85. 1% even when 60% of the propeller is occluded and can run at upto 35Hz on a 2W power budget.
SpikeMS: Deep Spiking Neural Network for Motion Segmentation
Spiking Neural Networks (SNN) are the so-called third generation of neural networks which attempt to more closely match the functioning of the biological brain.
Forecasting Action through Contact Representations from First Person Video
Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is demonstrated by pioneering work in cognitive science.
MorphEyes: Variable Baseline Stereo For Quadrotor Navigation
To our knowledge, this is the first work that applies the concept of morphable design to achieve a variable baseline stereo vision system on a quadrotor.
Grasping in the Dark: Zero-Shot Object Grasping Using Tactile Feedback
Several end-effector designs for robust manipulation have been proposed but they mostly work when provided with prior information about the objects or equipped with external sensors for estimating object shape or size.
Robotics
Hybrid Backpropagation Parallel Reservoir Networks
In many real-world applications, fully-differentiable RNNs such as LSTMs and GRUs have been widely deployed to solve time series learning tasks.
Deep Reservoir Networks with Learned Hidden Reservoir Weights using Direct Feedback Alignment
Deep Reservoir Computing has emerged as a new paradigm for deep learning, which is based around the reservoir computing principle of maintaining random pools of neurons combined with hierarchical deep learning.
A Deep 2-Dimensional Dynamical Spiking Neuronal Network for Temporal Encoding trained with STDP
Here we show that a large, deep layered SNN with dynamical, chaotic activity mimicking the mammalian cortex with biologically-inspired learning rules, such as STDP, is capable of encoding information from temporal data.
PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry
Odometry on aerial robots has to be of low latency and high robustness whilst also respecting the Size, Weight, Area and Power (SWAP) constraints as demanded by the size of the robot.
0-MMS: Zero-Shot Multi-Motion Segmentation With A Monocular Event Camera
Segmentation of moving objects in dynamic scenes is a key process in scene understanding for navigation tasks.
Egocentric Object Manipulation Graphs
We introduce Egocentric Object Manipulation Graphs (Ego-OMG) - a novel representation for activity modeling and anticipation of near future actions integrating three components: 1) semantic temporal structure of activities, 2) short-term dynamics, and 3) representations for appearance.
Following Instructions by Imagining and Reaching Visual Goals
While traditional methods for instruction-following typically assume prior linguistic and perceptual knowledge, many recent works in reinforcement learning (RL) have proposed learning policies end-to-end, typically by training neural networks to map joint representations of observations and instructions directly to actions.
EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras
To our knowledge, this is the first deep learning -- based solution to the problem of dynamic obstacle avoidance using event cameras on a quadrotor.
Network Deconvolution
Convolution is a central operation in Convolutional Neural Networks (CNNs), which applies a kernel to overlapping regions shifted across the image.
EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras
In addition to camera egomotion and a dense depth map, the network estimates pixel-wise independently moving object segmentation and computes per-object 3D translational velocities for moving objects.
Topology-Aware Non-Rigid Point Cloud Registration
In this paper, we introduce a non-rigid registration pipeline for pairs of unorganized point clouds that may be topologically different.
Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
In this work we present a lightweight, unsupervised learning pipeline for \textit{dense} depth, optical flow and egomotion estimation from sparse event output of the Dynamic Vision Sensor (DVS).
Extracting Contact and Motion from Manipulation Videos
When we physically interact with our environment using our hands, we touch objects and force them to move: contact and motion are defining properties of manipulation.
Evenly Cascaded Convolutional Networks
We introduce Evenly Cascaded convolutional Network (ECN), a neural network taking inspiration from the cascade algorithm of wavelet analysis.
cilantro: A Lean, Versatile, and Efficient Library for Point Cloud Data Processing
We introduce cilantro, an open-source C++ library for geometric and general-purpose point cloud data processing.
A Computational Theory for Life-Long Learning of Semantics
Semantic vectors are learned from data to express semantic relationships between elements of information, for the purpose of solving and informing downstream tasks.
Joint direct estimation of 3D geometry and 3D motion using spatio temporal gradients
Conventional image motion based structure from motion methods first compute optical flow, then solve for the 3D motion parameters based on the epipolar constraint, and finally recover the 3D geometry of the scene.
Event-based Moving Object Detection and Tracking
Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), are ideally suited for real-time motion analysis.
SalientDSO: Bringing Attention to Direct Sparse Odometry
We merge the successes of these two communities and present a way to incorporate semantic information in the form of visual saliency to Direct Sparse Odometry - a highly successful direct sparse VO algorithm.
GapFlyt: Active Vision Based Minimalist Structure-less Gap Detection For Quadrotor Flight
We use this philosophy to design a minimalist sensori-motor framework for a quadrotor to fly though unknown gaps without a 3D reconstruction of the scene using only a monocular camera and onboard sensing.
Robotics
Generalization of Learning using Reservoir Computing
Thus, as opposed to training the entire high dimensional reservoir state, the reservoir only needs to train on these unique relationships, allowing the reservoir to perform well with very few training examples.
On the Importance of Consistency in Training Deep Neural Networks
We conclude this paper with the construction of a novel contractive neural network.
Answering Image Riddles using Vision and Reasoning through Probabilistic Soft Logic
We compile a dataset of over 3k riddles where each riddle consists of 4 images and a groundtruth answer.
LightNet: A Versatile, Standalone Matlab-based Environment for Deep Learning
LightNet is a lightweight, versatile and purely Matlab-based deep learning framework.
Revisiting Active Perception
Despite the recent successes in robotics, artificial intelligence and computer vision, a complete artificial agent necessarily must include active perception.
What Can I Do Around Here? Deep Functional Scene Understanding for Cognitive Robots
For robots that have the capability to interact with the physical environment through their end effectors, understanding the surrounding scenes is not merely a task of image classification or object recognition.
Neural Self Talk: Image Understanding via Continuous Questioning and Answering
In this paper we consider the problem of continuously discovering image contents by actively asking image based questions and subsequently answering the questions being asked.
Learning the Semantics of Manipulation Action
In this paper we present a formal computational framework for modeling manipulation actions.
Detection and Segmentation of 2D Curved Reflection Symmetric Structures
Symmetry, as one of the key components of Gestalt theory, provides an important mid-level cue that serves as input to higher visual processes such as segmentation.
Contour Detection and Characterization for Asynchronous Event Sensors
The bio-inspired, asynchronous event-based dynamic vision sensor records temporal changes in the luminance of the scene at high temporal resolution.
From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge
Specifically, commonsense reasoning is applied on (a) detections obtained from existing perception methods on given images, (b) a "commonsense" knowledge base constructed using natural language processing of image annotations and (c) lexical ontological knowledge from resources such as WordNet.
Grasp Type Revisited: A Modern Perspective on a Classical Feature for Vision
The grasp type provides crucial information about human action.
Fast 2D Border Ownership Assignment
To this end, we use several different local cues: shape, spectral properties of boundary patches, and semi-global grouping cues that are indicative of perceived depth.
Detection of Manipulation Action Consequences (MAC)
There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions.
