Search Results for author: Kostas Daniilidis

Found 82 papers, 40 papers with code

Semantic keypoint-based pose estimation from single RGB frames

no code implementations12 Apr 2022 Karl Schmeckpeper, Philip R. Osteen, Yufu Wang, Georgios Pavlakos, Kenneth Chaney, Wyatt Jordan, Xiaowei Zhou, Konstantinos G. Derpanis, Kostas Daniilidis

Empirically, we show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios even against a cluttered background.

Pose Estimation

SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space

no code implementations5 Apr 2022 Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Edgar Dobriban, Kostas Daniilidis

In contrast to previous shape reconstruction methods that align the input to a regular grid, we operate directly on the irregular, unoriented point cloud.

CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism

no code implementations30 Mar 2022 Jiahui Lei, Kostas Daniilidis

While neural representations for static 3D shapes are widely studied, representations for deformable surfaces are limited to be template-dependent or lack efficiency.

Uncertainty-driven Planner for Exploration and Navigation

1 code implementation24 Feb 2022 Georgios Georgakis, Bernadette Bucher, Anton Arapin, Karl Schmeckpeper, Nikolai Matni, Kostas Daniilidis

We consider the problems of exploration and point-goal navigation in previously unseen environments, where the spatial complexity of indoor scenes and partial observability constitute these tasks challenging.

Learning Augmentation Distributions using Transformed Risk Minimization

no code implementations16 Nov 2021 Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Edgar Dobriban, Kostas Daniilidis

We theoretically compare TRM with standard risk minimization, and give a PAC-Bayes upper bound on its generalization error.

Rotated MNIST

Discovering and Achieving Goals via World Models

1 code implementation NeurIPS 2021 Russell Mendonca, Oleh Rybkin, Kostas Daniilidis, Danijar Hafner, Deepak Pathak

How can artificial agents learn to solve many diverse tasks in complex visual environments in the absence of any supervision?

Deep Semi-Supervised 3D Shape Reconstruction by Solving a Poisson Equation with Spectral Methods

no code implementations29 Sep 2021 Diego Patino, Carlos Esteves, Kostas Daniilidis

In this paper we propose a deep learning method for unsupervised 3D implicit shape reconstruction from point clouds.

3D Shape Reconstruction

Single-Camera 3D Head Fitting for Mixed Reality Clinical Applications

no code implementations6 Sep 2021 Tejas Mane, Aylar Bayramova, Kostas Daniilidis, Philippos Mordohai, Elena Bernardis

We address the problem of estimating the shape of a person's head, defined as the geometry of the complete head surface, from a video taken with a single moving camera, and determining the alignment of the fitted 3D head for all video frames, irrespective of the person's pose.

3D Reconstruction Face Reconstruction +1

Model-Based Reinforcement Learning via Latent-Space Collocation

1 code implementation24 Jun 2021 Oleh Rybkin, Chuning Zhu, Anusha Nagabandi, Kostas Daniilidis, Igor Mordatch, Sergey Levine

The resulting latent collocation method (LatCo) optimizes trajectories of latent states, which improves over previously proposed shooting methods for visual model-based RL on tasks with sparse rewards and long-term goals.

Model-based Reinforcement Learning reinforcement-learning

Discovering and Achieving Goals with World Models

no code implementations ICML Workshop URL 2021 Russell Mendonca, Oleh Rybkin, Kostas Daniilidis, Danijar Hafner, Deepak Pathak

How can an artificial agent learn to solve a wide range of tasks in a complex visual environment in the absence of external supervision?

Birds of a Feather: Capturing Avian Shape Models from Images

1 code implementation CVPR 2021 Yufu Wang, Nikos Kolotouros, Kostas Daniilidis, Marc Badger

We learn models of multiple species from the CUB dataset, and contribute new species-specific and multi-species shape models that are useful for downstream reconstruction tasks.

Object-centric Video Prediction without Annotation

1 code implementation6 May 2021 Karl Schmeckpeper, Georgios Georgakis, Kostas Daniilidis

Object-centric video prediction offers a solution to these problems by taking advantage of the simple prior that the world is made of objects and by providing a more natural interface for control.

Video Prediction

Deformable Linear Object Prediction Using Locally Linear Latent Dynamics

1 code implementation26 Mar 2021 Wenbo Zhang, Karl Schmeckpeper, Pratik Chaudhari, Kostas Daniilidis

We empirically demonstrate that our approach can predict the rope state accurately up to ten steps into the future and that our algorithm can find the optimal action given an initial state and a goal state.

Learning Portrait Style Representations

1 code implementation8 Dec 2020 Sadat Shaik, Bernadette Bucher, Nephele Agrafiotis, Stephen Phillips, Kostas Daniilidis, William Schmenner

We study style representations learned by neural network architectures incorporating these higher level characteristics.

Image Generation Zero-Shot Learning

Joint Estimation of Image Representations and their Lie Invariants

no code implementations5 Dec 2020 Christine Allen-Blanchette, Kostas Daniilidis

In both approaches, the underlying dynamics of the image sequence are modelled explicitly to disentangle them from the image representations.

Reinforcement Learning with Videos: Combining Offline Observations with Interaction

1 code implementation12 Nov 2020 Karl Schmeckpeper, Oleh Rybkin, Kostas Daniilidis, Sergey Levine, Chelsea Finn

In this paper, we consider the question: can we perform reinforcement learning directly on experience collected by humans?

reinforcement-learning

TLIO: Tight Learned Inertial Odometry

no code implementations6 Jul 2020 Wenxin Liu, David Caruso, Eddy Ilg, Jing Dong, Anastasios I. Mourikis, Kostas Daniilidis, Vijay Kumar, Jakob Engel

We show that our network, trained with pedestrian data from a headset, can produce statistically consistent measurement and uncertainty to be used as the update step in the filter, and the tightly-coupled system outperforms velocity integration approaches in position estimates, and AHRS attitude filter in orientation estimates.

Simple and Effective VAE Training with Calibrated Decoders

1 code implementation23 Jun 2020 Oleh Rybkin, Kostas Daniilidis, Sergey Levine

We perform the first comprehensive comparative analysis of calibrated decoder and provide recommendations for simple and effective VAE training.

Spin-Weighted Spherical CNNs

1 code implementation NeurIPS 2020 Carlos Esteves, Ameesh Makadia, Kostas Daniilidis

In this paper, we present a new type of spherical CNN that allows anisotropic filters in an efficient way, without ever leaving the spherical domain.

Semantic Segmentation

Coherent Reconstruction of Multiple Humans from a Single Image

1 code implementation CVPR 2020 Wen Jiang, Nikos Kolotouros, Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis

Our goal is to train a single network that learns to avoid these problems and generate a coherent 3D reconstruction of all the humans in the scene.

3D Depth Estimation 3D Human Reconstruction +4

Planning to Explore via Self-Supervised World Models

3 code implementations12 May 2020 Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak Pathak

Reinforcement learning allows solving complex tasks, however, the learning tends to be task-specific and the sample efficiency remains a challenge.

reinforcement-learning

Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks

2 code implementations ECCV 2020 Chankyu Lee, Adarsh Kumar Kosta, Alex Zihao Zhu, Kenneth Chaney, Kostas Daniilidis, Kaushik Roy

Spiking Neural Networks (SNNs) serve as ideal paradigms to handle event camera outputs, but deep SNNs suffer in terms of performance due to the spike vanishing phenomenon.

Frame Motion Detection +2

An Adversarial Objective for Scalable Exploration

1 code implementation13 Mar 2020 Bernadette Bucher, Karl Schmeckpeper, Nikolai Matni, Kostas Daniilidis

Model-based curiosity combines active learning approaches to optimal sampling with the information gain based incentives for exploration presented in the curiosity literature.

Active Learning

Reactive Navigation in Partially Familiar Planar Environments Using Semantic Perceptual Feedback

2 code implementations20 Feb 2020 Vasileios Vasilopoulos, Georgios Pavlakos, Karl Schmeckpeper, Kostas Daniilidis, Daniel E. Koditschek

This paper solves the planar navigation problem by recourse to an online reactive scheme that exploits recent advances in SLAM and visual object recognition to recast prior geometric knowledge in terms of an offline catalogue of familiar objects.

Robotics

Learning Predictive Models From Observation and Interaction

no code implementations ECCV 2020 Karl Schmeckpeper, Annie Xie, Oleh Rybkin, Stephen Tian, Kostas Daniilidis, Sergey Levine, Chelsea Finn

Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works, and then use this learned model to plan coordinated sequences of actions to bring about desired outcomes.

EventGAN: Leveraging Large Scale Image Datasets for Event Cameras

1 code implementation3 Dec 2019 Alex Zihao Zhu, ZiYun Wang, Kaung Khant, Kostas Daniilidis

Event cameras provide a number of benefits over traditional cameras, such as the ability to track incredibly fast motions, high dynamic range, and low power consumption.

Image Reconstruction Object Detection +2

TexturePose: Supervising Human Mesh Estimation with Texture Consistency

1 code implementation ICCV 2019 Georgios Pavlakos, Nikos Kolotouros, Kostas Daniilidis

Assuming that the texture of the person does not change dramatically between frames, we can apply a novel texture consistency loss, which enforces that each point in the texture map has the same texture value across all frames.

Pose Estimation Weakly-supervised 3D Human Pose Estimation

TagSLAM: Robust SLAM with Fiducial Markers

no code implementations1 Oct 2019 Bernd Pfrommer, Kostas Daniilidis

TagSLAM provides a convenient, flexible, and robust way of performing Simultaneous Localization and Mapping (SLAM) with AprilTag fiducial markers.

Camera Calibration Pose Estimation +2

Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop

1 code implementation ICCV 2019 Nikos Kolotouros, Georgios Pavlakos, Michael J. Black, Kostas Daniilidis

Our approach is self-improving by nature, since better network estimates can lead the optimization to better solutions, while more accurate optimization fits provide better supervision for the network.

3D Multi-Person Pose Estimation

Graph Neural Networks For Multi-Image Matching

no code implementations25 Sep 2019 Stephen Phillips, Kostas Daniilidis

In geometric computer vision applications, multi-image feature matching gives more accurate and robust solutions compared to simple two-image matching.

Graph Embedding

Keyframing the Future: Discovering Temporal Hierarchy with Keyframe-Inpainter Prediction

no code implementations25 Sep 2019 Karl Pertsch, Oleh Rybkin, Jingyun Yang, Konstantinos G. Derpanis, Kostas Daniilidis, Joseph J. Lim, Andrew Jaegle

To flexibly and efficiently reason about temporal sequences, abstract representations that compactly represent the important information in the sequence are needed.

Convolutional Mesh Regression for Single-Image Human Shape Reconstruction

1 code implementation CVPR 2019 Nikos Kolotouros, Georgios Pavlakos, Kostas Daniilidis

Image-based features are attached to the mesh vertices and the Graph-CNN is responsible to process them on the mesh structure, while the regression target for each vertex is its 3D location.

3D human pose and shape estimation Monocular 3D Human Pose Estimation +1

Event-based Vision: A Survey

4 code implementations17 Apr 2019 Guillermo Gallego, Tobi Delbruck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Joerg Conradt, Kostas Daniilidis, Davide Scaramuzza

Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur.

Event-based vision

Equivariant Multi-View Networks

1 code implementation ICCV 2019 Carlos Esteves, Yinshuang Xu, Christine Allen-Blanchette, Kostas Daniilidis

Several popular approaches to 3D vision tasks process multiple views of the input independently with deep neural networks pre-trained on natural images, achieving view permutation invariance through a single round of pooling over all views.

3D Shape Classification 3D Shape Retrieval +1

Motion Equivariant Networks for Event Cameras with the Temporal Normalization Transform

no code implementations18 Feb 2019 Alex Zihao Zhu, ZiYun Wang, Kostas Daniilidis

In this work, we propose a novel transformation for events from an event camera that is equivariant to optical flow under convolutions in the 3-D spatiotemporal domain.

Data Augmentation General Classification +1

All Graphs Lead to Rome: Learning Geometric and Cycle-Consistent Representations with Graph Convolutional Networks

1 code implementation7 Jan 2019 Stephen Phillips, Kostas Daniilidis

Image feature matching is a fundamental part of many geometric computer vision applications, and using multiple images can improve performance.

Graph Embedding

Monocular 3D Pose Recovery via Nonconvex Sparsity with Theoretical Analysis

no code implementations29 Dec 2018 Jianqiao Wangni, Dahua Lin, Ji Liu, Kostas Daniilidis, Jianbo Shi

For recovering 3D object poses from 2D images, a prevalent method is to pre-train an over-complete dictionary $\mathcal D=\{B_i\}_i^D$ of 3D basis poses.

Robustness Meets Deep Learning: An End-to-End Hybrid Pipeline for Unsupervised Learning of Egomotion

no code implementations20 Dec 2018 Alex Zihao Zhu, Wenxin Liu, ZiYun Wang, Vijay Kumar, Kostas Daniilidis

In this work, we propose a method that combines unsupervised deep learning predictions for optical flow and monocular disparity with a model based optimization procedure for instantaneous camera pose.

Optical Flow Estimation

Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

1 code implementation CVPR 2019 Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, Kostas Daniilidis

In this work, we propose a novel framework for unsupervised learning for event cameras that learns motion information from only the event stream.

Optical Flow Estimation

Cross-Domain 3D Equivariant Image Embeddings

1 code implementation6 Dec 2018 Carlos Esteves, Avneesh Sud, Zhengyi Luo, Kostas Daniilidis, Ameesh Makadia

This embedding encodes images with 3D shape properties and is equivariant to 3D rotations of the observed object.

3D Shape Classification Novel View Synthesis +1

Labeling Panoramas with Spherical Hourglass Networks

no code implementations6 Sep 2018 Carlos Esteves, Kostas Daniilidis, Ameesh Makadia

With the recent proliferation of consumer-grade 360{\deg} cameras, it is worth revisiting visual perception challenges with spherical cameras given the potential benefit of their global field of view.

Semantic Segmentation

Learning what you can do before doing anything

no code implementations ICLR 2019 Oleh Rybkin, Karl Pertsch, Konstantinos G. Derpanis, Kostas Daniilidis, Andrew Jaegle

We introduce a loss term that encourages the network to capture the composability of visual sequences and show that it leads to representations that disentangle the structure of actions.

Video Prediction

Learning to Estimate 3D Human Pose and Shape from a Single Color Image

no code implementations CVPR 2018 Georgios Pavlakos, Luyang Zhu, Xiaowei Zhou, Kostas Daniilidis

The proposed approach outperforms previous baselines on this task and offers an attractive solution for direct prediction of 3D shape from a single color image.

Human Motion Capture Using a Drone

1 code implementation17 Apr 2018 Xiaowei Zhou, Sikang Liu, Georgios Pavlakos, Vijay Kumar, Kostas Daniilidis

Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments.

Predicting the Future with Transformational States

no code implementations26 Mar 2018 Andrew Jaegle, Oleh Rybkin, Konstantinos G. Derpanis, Kostas Daniilidis

We couple this latent state with a recurrent neural network (RNN) core that predicts future frames by transforming past states into future states by applying the accumulated state transformation with a learned operator.

Realtime Time Synchronized Event-based Stereo

no code implementations ECCV 2018 Alex Zihao Zhu, Yibo Chen, Kostas Daniilidis

In this work, we propose a novel event based stereo method which addresses the problem of motion blur for a moving event camera.

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

2 code implementations19 Feb 2018 Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, Kostas Daniilidis

Event-based cameras have shown great promise in a variety of situations where frame based cameras suffer, such as high speed motions and high dynamic range scenes.

Frame Optical Flow Estimation

The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception

no code implementations30 Jan 2018 Alex Zihao Zhu, Dinesh Thakur, Tolga Ozaslan, Bernd Pfrommer, Vijay Kumar, Kostas Daniilidis

Event based cameras are a new passive sensing modality with a number of benefits over traditional cameras, including extremely low latency, asynchronous data acquisition, high dynamic range and very low power consumption.

Robotics

Fast, Autonomous Flight in GPS-Denied and Cluttered Environments

no code implementations6 Dec 2017 Kartik Mohta, Michael Watterson, Yash Mulgaonkar, Sikang Liu, Chao Qu, Anurag Makineni, Kelsey Saulnier, Ke Sun, Alex Zhu, Jeffrey Delmerico, Konstantinos Karydis, Nikolay Atanasov, Giuseppe Loianno, Davide Scaramuzza, Kostas Daniilidis, Camillo Jose Taylor, Vijay Kumar

One of the most challenging tasks for a flying robot is to autonomously navigate between target locations quickly and reliably while avoiding obstacles in its path, and with little to no a-priori knowledge of the operating environment.

Robotics

Multi-Image Semantic Matching by Mining Consistent Features

1 code implementation CVPR 2018 Qianqian Wang, Xiaowei Zhou, Kostas Daniilidis

This work proposes a multi-image matching method to estimate semantic correspondences across multiple images.

Graph Matching

Event-Based Visual Inertial Odometry

no code implementations CVPR 2017 Alex Zihao Zhu, Nikolay Atanasov, Kostas Daniilidis

An Extended Kalman Filter with a structureless measurement model then fuses the feature tracks with the output of the IMU.

6-DoF Object Pose from Semantic Keypoints

1 code implementation14 Mar 2017 Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G. Derpanis, Kostas Daniilidis

This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image.

Keypoint Detection Translation

Understanding image motion with group representations

no code implementations ICLR 2018 Andrew Jaegle, Stephen Phillips, Daphne Ippolito, Kostas Daniilidis

Our results demonstrate that this representation is useful for learning motion in the general setting where explicit labels are difficult to obtain.

Fast, Robust, Continuous Monocular Egomotion Computation

1 code implementation16 Feb 2016 Andrew Jaegle, Stephen Phillips, Kostas Daniilidis

We propose robust methods for estimating camera egomotion in noisy, real-world monocular image sequences in the general case of unknown observer rotation and translation with two views and a small baseline.

Motion Estimation Optical Flow Estimation +1

Single Image Pop-Up From Discriminatively Learned Parts

no code implementations ICCV 2015 Menglong Zhu, Xiaowei Zhou, Kostas Daniilidis

We introduce a new approach for estimating a fine grained 3D shape and continuous pose of an object from a single image.

Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach

no code implementations14 Sep 2015 Xiaowei Zhou, Menglong Zhu, Spyridon Leonardos, Kostas Daniilidis

We investigate the problem of estimating the 3D shape of an object defined by a set of 3D landmarks, given their 2D correspondences in a single image.

A Metric Parametrization for Trifocal Tensors With Non-Colinear Pinholes

no code implementations CVPR 2015 Spyridon Leonardos, Roberto Tron, Kostas Daniilidis

In this work, we investigate a new parametrization of the trifocal tensor for calibrated cameras with non-colinear pinholes obtained from a quotient Riemannian manifold.

Multi-Image Matching via Fast Alternating Minimization

no code implementations ICCV 2015 Xiaowei Zhou, Menglong Zhu, Kostas Daniilidis

In this paper we propose a global optimization-based approach to jointly matching a set of images.

Pose and Shape Estimation with Discriminatively Learned Parts

no code implementations1 Feb 2015 Menglong Zhu, Xiaowei Zhou, Kostas Daniilidis

We introduce a new approach for estimating the 3D pose and the 3D shape of an object from a single image.

3D Shape Estimation from 2D Landmarks: A Convex Relaxation Approach

no code implementations CVPR 2015 Xiaowei Zhou, Spyridon Leonardos, Xiaoyan Hu, Kostas Daniilidis

We investigate the problem of estimating the 3D shape of an object, given a set of 2D landmarks in a single image.

Geometric Urban Geo-Localization

no code implementations CVPR 2014 Mayank Bansal, Kostas Daniilidis

We propose a purely geometric correspondence-free approach to urban geo-localization using 3D point-ray features extracted from the Digital Elevation Map of an urban environment.

On the Quotient Representation for the Essential Manifold

no code implementations CVPR 2014 Roberto Tron, Kostas Daniilidis

The essential matrix, which encodes the epipolar constraint between points in two projective views, is a cornerstone of modern computer vision.

Geometric Polynomial Constraints in Higher-Order Graph Matching

no code implementations24 May 2014 Mayank Bansal, Kostas Daniilidis

In this paper, we address the problem of finding correspondences in the absence of unary or pairwise constraints as it emerges in problems where unary appearance similarity like SIFT matches is not available.

Graph Matching

Active Deformable Part Models

no code implementations1 Apr 2014 Menglong Zhu, Nikolay Atanasov, George J. Pappas, Kostas Daniilidis

This paper presents an active approach for part-based object detection, which optimizes the order of part filter evaluations and the time at which to stop and make a prediction.

General Classification Object Detection

Nonmyopic View Planning for Active Object Detection

no code implementations20 Sep 2013 Nikolay Atanasov, Bharath Sankaran, Jerome Le Ny, George J. Pappas, Kostas Daniilidis

One of the central problems in computer vision is the detection of semantically important objects and the estimation of their pose.

Active Object Detection Object Detection +1

Joint Spectral Correspondence for Disparate Image Matching

no code implementations CVPR 2013 Mayank Bansal, Kostas Daniilidis

We address the problem of matching images with disparate appearance arising from factors like dramatic illumination (day vs. night), age (historic vs. new) and rendering style differences.

Cannot find the paper you are looking for? You can Submit a new open access paper.