Search Results for author:
Found 29 papers, 3 papers with code
TimeRewind: Rewinding Time with Image-and-Events Video Diffusion
Through extensive experimentation, we demonstrate the capability of our approach to synthesize high-quality videos that effectively ``rewind'' time, showcasing the potential of combining event camera technology with generative models.
NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction
NatSGD serves as a foundational resource at the intersection of machine learning and HRI research, and we demonstrate its effectiveness in training robots to understand tasks through multimodal human commands, emphasizing the significance of jointly considering speech and gestures.
Event-based Vision for Early Prediction of Manipulation Actions
In this study, we introduce an event-based dataset on fine-grained manipulation actions and perform an experimental study on the use of transformers for action prediction with events.
Therbligs in Action: Video Understanding through Motion Primitives
In this paper we introduce a rule-based, compositional, and hierarchical modeling of action using Therbligs as our atoms.
Gluing Neural Networks Symbolically Through Hyperdimensional Computing
This process can be performed iteratively and even on single neural networks by instead making a consensus of multiple classification hypervectors.
Forecasting Action through Contact Representations from First Person Video
Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is demonstrated by pioneering work in cognitive science.
A Deep 2-Dimensional Dynamical Spiking Neuronal Network for Temporal Encoding trained with STDP
Here we show that a large, deep layered SNN with dynamical, chaotic activity mimicking the mammalian cortex with biologically-inspired learning rules, such as STDP, is capable of encoding information from temporal data.
Egocentric Object Manipulation Graphs
We introduce Egocentric Object Manipulation Graphs (Ego-OMG) - a novel representation for activity modeling and anticipation of near future actions integrating three components: 1) semantic temporal structure of activities, 2) short-term dynamics, and 3) representations for appearance.
Following Instructions by Imagining and Reaching Visual Goals
While traditional methods for instruction-following typically assume prior linguistic and perceptual knowledge, many recent works in reinforcement learning (RL) have proposed learning policies end-to-end, typically by training neural networks to map joint representations of observations and instructions directly to actions.
EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras
In addition to camera egomotion and a dense depth map, the network estimates pixel-wise independently moving object segmentation and computes per-object 3D translational velocities for moving objects.
Topology-Aware Non-Rigid Point Cloud Registration
In this paper, we introduce a non-rigid registration pipeline for pairs of unorganized point clouds that may be topologically different.
Extracting Contact and Motion from Manipulation Videos
When we physically interact with our environment using our hands, we touch objects and force them to move: contact and motion are defining properties of manipulation.
Real-time clustering and multi-target tracking using event-based sensors
The potential of our approach is demonstrated in a multi-target tracking application using Kalman filters to smooth the trajectories.
cilantro: A Lean, Versatile, and Efficient Library for Point Cloud Data Processing
We introduce cilantro, an open-source C++ library for geometric and general-purpose point cloud data processing.
Event-based Moving Object Detection and Tracking
Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), are ideally suited for real-time motion analysis.
On the Importance of Consistency in Training Deep Neural Networks
We conclude this paper with the construction of a novel contractive neural network.
Fast Task-Specific Target Detection via Graph Based Constraints Representation and Checking
In this work, we present a fast target detection framework for real-world robotics applications.
Co-active Learning to Adapt Humanoid Movement for Manipulation
It is designed to adapt the original imitation trajectories, which are learned from demonstrations, to novel situations with various constraints.
Reliable Attribute-Based Object Recognition Using High Predictive Value Classifiers
We consider the problem of object recognition in 3D using an ensemble of attribute-based classifiers.
LightNet: A Versatile, Standalone Matlab-based Environment for Deep Learning
LightNet is a lightweight, versatile and purely Matlab-based deep learning framework.
What Can I Do Around Here? Deep Functional Scene Understanding for Cognitive Robots
For robots that have the capability to interact with the physical environment through their end effectors, understanding the surrounding scenes is not merely a task of image classification or object recognition.
Neural Self Talk: Image Understanding via Continuous Questioning and Answering
In this paper we consider the problem of continuously discovering image contents by actively asking image based questions and subsequently answering the questions being asked.
Learning the Semantics of Manipulation Action
In this paper we present a formal computational framework for modeling manipulation actions.
Contour Detection and Characterization for Asynchronous Event Sensors
The bio-inspired, asynchronous event-based dynamic vision sensor records temporal changes in the luminance of the scene at high temporal resolution.
Detection and Segmentation of 2D Curved Reflection Symmetric Structures
Symmetry, as one of the key components of Gestalt theory, provides an important mid-level cue that serves as input to higher visual processes such as segmentation.
From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge
Specifically, commonsense reasoning is applied on (a) detections obtained from existing perception methods on given images, (b) a "commonsense" knowledge base constructed using natural language processing of image annotations and (c) lexical ontological knowledge from resources such as WordNet.
Grasp Type Revisited: A Modern Perspective on a Classical Feature for Vision
The grasp type provides crucial information about human action.
Fast 2D Border Ownership Assignment
To this end, we use several different local cues: shape, spectral properties of boundary patches, and semi-global grouping cues that are indicative of perceived depth.
Detection of Manipulation Action Consequences (MAC)
There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions.
