Search Results for author: Cornelia Fermuller

Found 32 papers, 4 papers with code

Diving Deep into the Motion Representation of Video-Text Models

no code implementations7 Jun 2024 Chinmaya Devaraj, Cornelia Fermuller, Yiannis Aloimonos

This method proves to be effective on two action datasets for the motion description retrieval task.

Retrieval Text Retrieval +1

Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion

no code implementations5 Jun 2024 Tianyi Xiong, Jiayi Wu, Botao He, Cornelia Fermuller, Yiannis Aloimonos, Heng Huang, Christopher A. Metzler

By combining differentiable rendering with explicit point-based scene representations, 3D Gaussian Splatting (3DGS) has demonstrated breakthrough 3D reconstruction capabilities.

3D Reconstruction 3D Scene Reconstruction +1

TimeRewind: Rewinding Time with Image-and-Events Video Diffusion

no code implementations20 Mar 2024 Jingxi Chen, Brandon Y. Feng, Haoming Cai, Mingyang Xie, Christopher Metzler, Cornelia Fermuller, Yiannis Aloimonos

Through extensive experimentation, we demonstrate the capability of our approach to synthesize high-quality videos that effectively ``rewind'' time, showcasing the potential of combining event camera technology with generative models.

NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction

no code implementations4 Mar 2024 Snehesh Shrestha, Yantian Zha, Saketh Banagiri, Ge Gao, Yiannis Aloimonos, Cornelia Fermuller

NatSGD serves as a foundational resource at the intersection of machine learning and HRI research, and we demonstrate its effectiveness in training robots to understand tasks through multimodal human commands, emphasizing the significance of jointly considering speech and gestures.

Event-based Vision for Early Prediction of Manipulation Actions

1 code implementation26 Jul 2023 Daniel Deniz, Cornelia Fermuller, Eduardo Ros, Manuel Rodriguez-Alvarez, Francisco Barranco

In this study, we introduce an event-based dataset on fine-grained manipulation actions and perform an experimental study on the use of transformers for action prediction with events.

Action Recognition Data Compression +1

Gluing Neural Networks Symbolically Through Hyperdimensional Computing

no code implementations31 May 2022 Peter Sutor, Dehao Yuan, Douglas Summers-Stay, Cornelia Fermuller, Yiannis Aloimonos

This process can be performed iteratively and even on single neural networks by instead making a consensus of multiple classification hypervectors.

Forecasting Action through Contact Representations from First Person Video

no code implementations1 Feb 2021 Eadom Dessalene, Chinmaya Devaraj, Michael Maynord, Cornelia Fermuller, Yiannis Aloimonos

Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is demonstrated by pioneering work in cognitive science.

Action Anticipation Object

A Deep 2-Dimensional Dynamical Spiking Neuronal Network for Temporal Encoding trained with STDP

no code implementations1 Sep 2020 Matthew Evanusa, Cornelia Fermuller, Yiannis Aloimonos

Here we show that a large, deep layered SNN with dynamical, chaotic activity mimicking the mammalian cortex with biologically-inspired learning rules, such as STDP, is capable of encoding information from temporal data.

Egocentric Object Manipulation Graphs

no code implementations5 Jun 2020 Eadom Dessalene, Michael Maynord, Chinmaya Devaraj, Cornelia Fermuller, Yiannis Aloimonos

We introduce Egocentric Object Manipulation Graphs (Ego-OMG) - a novel representation for activity modeling and anticipation of near future actions integrating three components: 1) semantic temporal structure of activities, 2) short-term dynamics, and 3) representations for appearance.

Action Anticipation Attribute +1

Following Instructions by Imagining and Reaching Visual Goals

no code implementations25 Jan 2020 John Kanu, Eadom Dessalene, Xiaomin Lin, Cornelia Fermuller, Yiannis Aloimonos

While traditional methods for instruction-following typically assume prior linguistic and perceptual knowledge, many recent works in reinforcement learning (RL) have proposed learning policies end-to-end, typically by training neural networks to map joint representations of observations and instructions directly to actions.

Instruction Following Reinforcement Learning (RL)

EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras

no code implementations18 Mar 2019 Anton Mitrokhin, Chengxi Ye, Cornelia Fermuller, Yiannis Aloimonos, Tobi Delbruck

In addition to camera egomotion and a dense depth map, the network estimates pixel-wise independently moving object segmentation and computes per-object 3D translational velocities for moving objects.

Motion Segmentation Object +1

Topology-Aware Non-Rigid Point Cloud Registration

no code implementations16 Nov 2018 Konstantinos Zampogiannis, Cornelia Fermuller, Yiannis Aloimonos

In this paper, we introduce a non-rigid registration pipeline for pairs of unorganized point clouds that may be topologically different.

Dynamic Reconstruction Event Detection +2

Extracting Contact and Motion from Manipulation Videos

no code implementations13 Jul 2018 Konstantinos Zampogiannis, Kanishka Ganguly, Cornelia Fermuller, Yiannis Aloimonos

When we physically interact with our environment using our hands, we touch objects and force them to move: contact and motion are defining properties of manipulation.

Clustering Imitation Learning +2

Real-time clustering and multi-target tracking using event-based sensors

no code implementations8 Jul 2018 Francisco Barranco, Cornelia Fermuller, Eduardo Ros

The potential of our approach is demonstrated in a multi-target tracking application using Kalman filters to smooth the trajectories.

Clustering Event-based vision +2

Event-based Moving Object Detection and Tracking

no code implementations12 Mar 2018 Anton Mitrokhin, Cornelia Fermuller, Chethan Parameshwara, Yiannis Aloimonos

Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), are ideally suited for real-time motion analysis.

Event-based vision Motion Detection +5

On the Importance of Consistency in Training Deep Neural Networks

no code implementations2 Aug 2017 Chengxi Ye, Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos

We conclude this paper with the construction of a novel contractive neural network.

Co-active Learning to Adapt Humanoid Movement for Manipulation

no code implementations12 Sep 2016 Ren Mao, John S. Baras, Yezhou Yang, Cornelia Fermuller

It is designed to adapt the original imitation trajectories, which are learned from demonstrations, to novel situations with various constraints.

Active Learning

LightNet: A Versatile, Standalone Matlab-based Environment for Deep Learning

1 code implementation9 May 2016 Chengxi Ye, Chen Zhao, Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos

LightNet is a lightweight, versatile and purely Matlab-based deep learning framework.

What Can I Do Around Here? Deep Functional Scene Understanding for Cognitive Robots

no code implementations29 Jan 2016 Chengxi Ye, Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos

For robots that have the capability to interact with the physical environment through their end effectors, understanding the surrounding scenes is not merely a task of image classification or object recognition.

Image Classification Object Recognition +1

Neural Self Talk: Image Understanding via Continuous Questioning and Answering

no code implementations10 Dec 2015 Yezhou Yang, Yi Li, Cornelia Fermuller, Yiannis Aloimonos

In this paper we consider the problem of continuously discovering image contents by actively asking image based questions and subsequently answering the questions being asked.

Question Answering Question Generation +2

Detection and Segmentation of 2D Curved Reflection Symmetric Structures

no code implementations ICCV 2015 Ching L. Teo, Cornelia Fermuller, Yiannis Aloimonos

Symmetry, as one of the key components of Gestalt theory, provides an important mid-level cue that serves as input to higher visual processes such as segmentation.

Segmentation Symmetry Detection

Contour Detection and Characterization for Asynchronous Event Sensors

no code implementations ICCV 2015 Francisco Barranco, Ching L. Teo, Cornelia Fermuller, Yiannis Aloimonos

The bio-inspired, asynchronous event-based dynamic vision sensor records temporal changes in the luminance of the scene at high temporal resolution.

Boundary Detection Contour Detection +2

From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge

no code implementations10 Nov 2015 Somak Aditya, Yezhou Yang, Chitta Baral, Cornelia Fermuller, Yiannis Aloimonos

Specifically, commonsense reasoning is applied on (a) detections obtained from existing perception methods on given images, (b) a "commonsense" knowledge base constructed using natural language processing of image annotations and (c) lexical ontological knowledge from resources such as WordNet.

image-sentence alignment Sentence

Fast 2D Border Ownership Assignment

no code implementations CVPR 2015 Ching Teo, Cornelia Fermuller, Yiannis Aloimonos

To this end, we use several different local cues: shape, spectral properties of boundary patches, and semi-global grouping cues that are indicative of perceived depth.

Boundary Detection

Detection of Manipulation Action Consequences (MAC)

no code implementations CVPR 2013 Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos

There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions.

Action Recognition Temporal Action Localization

Cannot find the paper you are looking for? You can Submit a new open access paper.