Search Results for author: Dieter Fox

Found 99 papers, 43 papers with code

Factory: Fast Contact for Robotic Assembly

no code implementations7 May 2022 Yashraj Narang, Kier Storey, Iretiayo Akinola, Miles Macklin, Philipp Reist, Lukasz Wawrzyniak, Yunrong Guo, Adam Moravanszky, Gavriel State, Michelle Lu, Ankur Handa, Dieter Fox

We aim for Factory to open the doors to using simulation for robotic assembly, as well as many other contact-rich applications in robotics.

Model Predictive Control for Fluid Human-to-Robot Handovers

no code implementations31 Mar 2022 Wei Yang, Balakumar Sundaralingam, Chris Paxton, Iretiayo Akinola, Yu-Wei Chao, Maya Cakmak, Dieter Fox

However, how to responsively generate smooth motions to take an object from a human is still an open question.

IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

no code implementations1 Feb 2022 Ankit Goyal, Arsalan Mousavian, Chris Paxton, Yu-Wei Chao, Brian Okorn, Jia Deng, Dieter Fox

Accurate object rearrangement from vision is a crucial problem for a wide variety of real-world robotics applications in unstructured environments.

Optical Flow Estimation

iCaps: Iterative Category-level Object Pose and Shape Estimation

1 code implementation31 Dec 2021 Xinke Deng, Junyi Geng, Timothy Bretl, Yu Xiang, Dieter Fox

The auto-encoder can be used in a particle filter framework to estimate and track 6D poses of objects in a category.

Learning Perceptual Concepts by Bootstrapping from Human Queries

no code implementations9 Nov 2021 Andreea Bobu, Chris Paxton, Wei Yang, Balakumar Sundaralingam, Yu-Wei Chao, Maya Cakmak, Dieter Fox

To address this challenge, we propose a new approach whereby the robot learns a low-dimensional variant of the concept and uses it to generate a larger data set for learning the concept in the high-dimensional space.

Motion Planning

StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects

no code implementations19 Oct 2021 Weiyu Liu, Chris Paxton, Tucker Hermans, Dieter Fox

Geometric organization of objects into semantically meaningful arrangements pervades the built world.

Continuous-Time Fitted Value Iteration for Robust Policies

1 code implementation5 Oct 2021 Michael Lutter, Boris Belousov, Shie Mannor, Dieter Fox, Animesh Garg, Jan Peters

Especially for continuous control, solving this differential equation and its extension the Hamilton-Jacobi-Isaacs equation, is important as it yields the optimal policy that achieves the maximum reward on a give task.

Continuous Control

Generalizing Successor Features to continuous domains for Multi-task Learning

no code implementations29 Sep 2021 Melissa Mozifian, Dieter Fox, David Meger, Fabio Ramos, Animesh Garg

In this paper, we consider the problem of continuous control for various robot manipulation tasks with an explicit representation that promotes skill reuse while learning multiple tasks, related through the reward function.

Continuous Control Decision Making +1

CLIPort: What and Where Pathways for Robotic Manipulation

1 code implementation24 Sep 2021 Mohit Shridhar, Lucas Manuelli, Dieter Fox

We even learn one multi-task policy for 10 simulated and 9 real-world tasks that is better or comparable to single-task policies.

Imitation Learning Robotic Grasping

SORNet: Spatial Object-Centric Representations for Sequential Manipulation

1 code implementation8 Sep 2021 Wentao Yuan, Chris Paxton, Karthik Desingh, Dieter Fox

Sequential manipulation tasks require a robot to perceive the state of an environment and plan a sequence of actions leading to a desired goal state, where the ability to reason about spatial relationships among object entities from raw sensor inputs is crucial.

Predicting Stable Configurations for Semantic Placement of Novel Objects

1 code implementation26 Aug 2021 Chris Paxton, Chris Xie, Tucker Hermans, Dieter Fox

We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.

Motion Planning

A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution

no code implementations12 Jul 2021 Valts Blukis, Chris Paxton, Dieter Fox, Animesh Garg, Yoav Artzi

Natural language provides an accessible and expressive interface to specify long-term tasks for robotic agents.

BayesSimIG: Scalable Parameter Inference for Adaptive Domain Randomization with IsaacGym

1 code implementation9 Jul 2021 Rika Antonova, Fabio Ramos, Rafael Possas, Dieter Fox

This paper outlines BayesSimIG: a library that provides an implementation of BayesSim integrated with the recently released NVIDIA IsaacGym.

reinforcement-learning

LanguageRefer: Spatial-Language Model for 3D Visual Grounding

no code implementations7 Jul 2021 Junha Roh, Karthik Desingh, Ali Farhadi, Dieter Fox

Specifically, given a reconstructed 3D scene in the form of point clouds with 3D bounding boxes of potential object candidates, and a language utterance referring to a target object in the scene, our model successfully identifies the target object from a set of potential candidates.

Language Modelling Visual Grounding

RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks

1 code implementation29 Jun 2021 Christopher Xie, Arsalan Mousavian, Yu Xiang, Dieter Fox

We postulate that a network architecture that encodes relations between objects at a high-level can be beneficial.

NeRP: Neural Rearrangement Planning for Unknown Objects

no code implementations2 Jun 2021 Ahmed H. Qureshi, Arsalan Mousavian, Chris Paxton, Michael C. Yip, Dieter Fox

We propose NeRP (Neural Rearrangement Planning), a deep learning based approach for multi-step neural object rearrangement planning which works with never-before-seen objects, that is trained on simulation data, and generalizes to the real world.

Robust Value Iteration for Continuous Control Tasks

1 code implementation25 May 2021 Michael Lutter, Shie Mannor, Jan Peters, Dieter Fox, Animesh Garg

The adversarial perturbations encourage a optimal policy that is robust to changes in the dynamics.

Continuous Control reinforcement-learning

Value Iteration in Continuous Actions, States and Time

1 code implementation10 May 2021 Michael Lutter, Shie Mannor, Jan Peters, Dieter Fox, Animesh Garg

This algorithm enables dynamic programming for continuous states and actions with a known dynamics model.

RGB-D Local Implicit Function for Depth Completion of Transparent Objects

1 code implementation CVPR 2021 Luyang Zhu, Arsalan Mousavian, Yu Xiang, Hammad Mazhar, Jozef van Eenbergen, Shoubhik Debnath, Dieter Fox

Key to our approach is a local implicit neural representation built on ray-voxel pairs that allows our method to generalize to unseen objects and achieve fast inference speed.

Depth Completion Depth Estimation +1

Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes

1 code implementation25 Mar 2021 Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, Dieter Fox

Our novel grasp representation treats 3D points of the recorded point cloud as potential grasp contacts.

Grasp Generation Robotic Grasping

Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees

no code implementations24 Dec 2020 M. Asif Rana, Anqi Li, Dieter Fox, Sonia Chernova, Byron Boots, Nathan Ratliff

The policy structure provides the user an interface to 1) specifying the spaces that are directly relevant to the completion of the tasks, and 2) designing policies for certain tasks that do not need to be learned.

Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshop

no code implementations7 Dec 2020 Sebastian Höfer, Kostas Bekris, Ankur Handa, Juan Camilo Gamboa, Florian Golemo, Melissa Mozifian, Chris Atkeson, Dieter Fox, Ken Goldberg, John Leonard, C. Karen Liu, Jan Peters, Shuran Song, Peter Welinder, Martha White

This report presents the debates, posters, and discussions of the Sim2Real workshop held in conjunction with the 2020 edition of the "Robotics: Science and System" conference.

Object Rearrangement Using Learned Implicit Collision Functions

1 code implementation21 Nov 2020 Michael Danielczuk, Arsalan Mousavian, Clemens Eppner, Dieter Fox

The learned model outperforms both traditional pipelines and learned ablations by 9. 8% in accuracy on a dataset of simulated collision queries and is 75x faster than the best-performing baseline.

ACRONYM: A Large-Scale Grasp Dataset Based on Simulation

1 code implementation18 Nov 2020 Clemens Eppner, Arsalan Mousavian, Dieter Fox

We introduce ACRONYM, a dataset for robot grasp planning based on physics simulation.

Reactive Human-to-Robot Handovers of Arbitrary Objects

no code implementations17 Nov 2020 Wei Yang, Chris Paxton, Arsalan Mousavian, Yu-Wei Chao, Maya Cakmak, Dieter Fox

We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects, a user study with naive users (N=6) handing over a subset of 15 objects, and a systematic evaluation examining different ways of handing objects.

Grasp Generation Motion Planning

Reactive Long Horizon Task Execution via Visual Skill and Precondition Models

no code implementations17 Nov 2020 Shohin Mukherjee, Chris Paxton, Arsalan Mousavian, Adam Fishman, Maxim Likhachev, Dieter Fox

Zero-shot execution of unseen robotic tasks is important to allowing robots to perform a wide variety of tasks in human environments, but collecting the amounts of data necessary to train end-to-end policies in the real-world is often infeasible.

Stein Variational Model Predictive Control

no code implementations15 Nov 2020 Alexander Lambert, Adam Fishman, Dieter Fox, Byron Boots, Fabio Ramos

By casting MPC as a Bayesian inference problem, we employ variational methods for posterior computation, naturally encoding the complexity and multi-modality of the decision making problem.

Bayesian Inference Decision Making +1

Multimodal Trajectory Prediction via Topological Invariance for Navigation at Uncontrolled Intersections

1 code implementation8 Nov 2020 Junha Roh, Christoforos Mavrogiannis, Rishabh Madan, Dieter Fox, Siddhartha S. Srinivasa

Our key insight is that the geometric structure of the intersection and the incentive of agents to move efficiently and avoid collisions (rationality) reduces the space of likely behaviors, effectively relaxing the problem of trajectory prediction.

Trajectory Prediction

Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds

1 code implementation2 Oct 2020 Lirui Wang, Yu Xiang, Wei Yang, Arsalan Mousavian, Dieter Fox

We demonstrate that our learned policy can be integrated into a tabletop 6D grasping system and a human-robot handover system to improve the grasping performance of unseen objects.

Imitation Learning Motion Planning +1

Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation

1 code implementation30 Jul 2020 Yu Xiang, Christopher Xie, Arsalan Mousavian, Dieter Fox

In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data.

Metric Learning Semantic Segmentation +1

Causal Discovery in Physical Systems from Videos

1 code implementation NeurIPS 2020 Yunzhu Li, Antonio Torralba, Animashree Anandkumar, Dieter Fox, Animesh Garg

We assume access to different configurations and environmental conditions, i. e., data from unknown interventions on the underlying system; thus, we can hope to discover the correct underlying causal graph without explicit interventions.

Causal Discovery

Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems

2 code implementations L4DC 2020 Muhammad Asif Rana, Anqi Li, Dieter Fox, Byron Boots, Fabio Ramos, Nathan Ratliff

The complex motions are encoded as rollouts of a stable dynamical system, which, under a change of coordinates defined by a diffeomorphism, is equivalent to a simple, hand-specified dynamical system.

Density Estimation

Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning

no code implementations21 May 2020 Michelle A. Lee, Carlos Florensa, Jonathan Tremblay, Nathan Ratliff, Animesh Garg, Fabio Ramos, Dieter Fox

Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state.

Watching the World Go By: Representation Learning from Unlabeled Videos

1 code implementation18 Mar 2020 Daniel Gordon, Kiana Ehsani, Dieter Fox, Ali Farhadi

Recent single image unsupervised representation learning techniques show remarkable success on a variety of tasks.

Data Augmentation Representation Learning

Human Grasp Classification for Reactive Human-to-Robot Handovers

no code implementations12 Mar 2020 Wei Yang, Chris Paxton, Maya Cakmak, Dieter Fox

In this paper, we propose an approach for human-to-robot handovers in which the robot meets the human halfway, by classifying the human's grasp of the object and quickly planning a trajectory accordingly to take the object from the human's hand according to their intent.

Classification General Classification

Transferable Task Execution from Pixels through Deep Planning Domain Learning

no code implementations8 Mar 2020 Kei Kase, Chris Paxton, Hammad Mazhar, Tetsuya OGATA, Dieter Fox

On the other hand, symbolic planning methods such as STRIPS have long been able to solve new problems given only a domain definition and a symbolic goal, but these approaches often struggle on the real world robotic tasks due to the challenges of grounding these symbols from sensor data in a partially-observable world.

Information Theoretic Model Predictive Q-Learning

no code implementations31 Dec 2019 Mohak Bhardwaj, Ankur Handa, Dieter Fox, Byron Boots

Model-free Reinforcement Learning (RL) works well when experience can be collected cheaply and model-based RL is effective when system dynamics can be modeled accurately.

Decision Making Q-Learning +1

A Billion Ways to Grasp: An Evaluation of Grasp Sampling Schemes on a Dense, Physics-based Grasp Data Set

no code implementations11 Dec 2019 Clemens Eppner, Arsalan Mousavian, Dieter Fox

With the increasing speed and quality of physics simulations, generating large-scale grasping data sets that feed learning algorithms is becoming more and more popular.

ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

5 code implementations CVPR 2020 Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, Dieter Fox

We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks.

Natural Language Visual Grounding

Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection

1 code implementation22 Nov 2019 Lirui Wang, Yu Xiang, Dieter Fox

In robot manipulation, planning the motion of a robot manipulator to grasp an object is a fundamental problem.

Robotics

Camera-to-Robot Pose Estimation from a Single Image

2 code implementations21 Nov 2019 Timothy E. Lee, Jonathan Tremblay, Thang To, Jia Cheng, Terry Mosier, Oliver Kroemer, Dieter Fox, Stan Birchfield

We show experimental results for three different camera sensors, demonstrating that our approach is able to achieve accuracy with a single frame that is better than that of classic off-line hand-eye calibration using multiple frames.

Robotics

Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies

no code implementations21 Nov 2019 Visak Kumar, Tucker Herman, Dieter Fox, Stan Birchfield, Jonathan Tremblay

We propose a Grasping Objects Approach for Tactile (GOAT) robotic hands, learning to overcome the reality gap problem.

Robotics

Motion Reasoning for Goal-Based Imitation Learning

no code implementations13 Nov 2019 De-An Huang, Yu-Wei Chao, Chris Paxton, Xinke Deng, Li Fei-Fei, Juan Carlos Niebles, Animesh Garg, Dieter Fox

We further show that by using the automatically inferred goal from the video demonstration, our robot is able to reproduce the same task in a real kitchen environment.

Imitation Learning Motion Planning

IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data

no code implementations13 Nov 2019 Ajay Mandlekar, Fabio Ramos, Byron Boots, Silvio Savarese, Li Fei-Fei, Animesh Garg, Dieter Fox

For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task.

Online Replanning in Belief Space for Partially Observable Task and Motion Problems

1 code implementation11 Nov 2019 Caelan Reed Garrett, Chris Paxton, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox

To solve multi-step manipulation tasks in the real world, an autonomous robot must take actions to observe its environment and react to unexpected observations.

Continuous Control

Motion-Nets: 6D Tracking of Unknown Objects in Unseen Environments using RGB

no code implementations30 Oct 2019 Felix Leeb, Arunkumar Byravan, Dieter Fox

In this work, we bridge the gap between recent pose estimation and tracking work to develop a powerful method for robots to track objects in their surroundings.

Pose Estimation Translation

Conditional Driving from Natural Language Instructions

no code implementations16 Oct 2019 Junha Roh, Chris Paxton, Andrzej Pronobis, Ali Farhadi, Dieter Fox

Widespread adoption of self-driving cars will depend not only on their safety but largely on their ability to interact with human users.

Imitation Learning Self-Driving Cars

DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System

no code implementations7 Oct 2019 Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff, Dieter Fox

Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks.

Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping

no code implementations7 Oct 2019 Mustafa Mukadam, Ching-An Cheng, Dieter Fox, Byron Boots, Nathan Ratliff

RMPfusion supplements RMPflow with weight functions that can hierarchically reshape the Lyapunov functions of the subtask RMPs according to the current configuration of the robot and environment.

Imitation Learning

Self-supervised 6D Object Pose Estimation for Robot Manipulation

3 code implementations23 Sep 2019 Xinke Deng, Yu Xiang, Arsalan Mousavian, Clemens Eppner, Timothy Bretl, Dieter Fox

In this way, our system is able to continuously collect data and improve its pose estimation modules.

Robotics

Part Segmentation for Highly Accurate Deformable Tracking in Occlusions via Fully Convolutional Neural Networks

no code implementations5 Aug 2019 Weilin Wan, Aaron Walsman, Dieter Fox

While recent work has shown direct estimation techniques can be quite powerful, geometric tracking methods using point clouds can provide a very high level of 3D accuracy which is necessary for many robotic applications.

Data Augmentation Frame +1

The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation

no code implementations30 Jul 2019 Christopher Xie, Yu Xiang, Arsalan Mousavian, Dieter Fox

We show that our method, trained on this dataset, can produce sharp and accurate masks, outperforming state-of-the-art methods on unseen object instance segmentation.

Semantic Segmentation Unseen Object Instance Segmentation

BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators

1 code implementation4 Jun 2019 Fabio Ramos, Rafael Carvalhaes Possas, Dieter Fox

We introduce BayesSim, a framework for robotics simulations allowing a full Bayesian treatment for the parameters of the simulator.

Motion Planning

PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking

1 code implementation22 May 2019 Xinke Deng, Arsalan Mousavian, Yu Xiang, Fei Xia, Timothy Bretl, Dieter Fox

In this work, we formulate the 6D object pose tracking problem in the Rao-Blackwellized particle filtering framework, where the 3D rotation and the 3D translation of an object are decoupled.

6D Pose Estimation 6D Pose Estimation using RGB +2

ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact

4 code implementations7 Apr 2019 Samarth Brahmbhatt, Ankur Handa, James Hays, Dieter Fox

Using a dataset of contact demonstrations from humans grasping diverse household objects, we synthesize functional grasps for three hand models and two functional intents.

Prospection: Interpretable Plans From Language By Predicting the Future

no code implementations20 Mar 2019 Chris Paxton, Yonatan Bisk, Jesse Thomason, Arunkumar Byravan, Dieter Fox

High-level human instructions often correspond to behaviors with multiple implicit steps.

What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planning

no code implementations6 Jan 2019 Daniel Gordon, Dieter Fox, Ali Farhadi

In this work we propose Hierarchical Planning and Reinforcement Learning (HIP-RL), a method for merging the benefits and capabilities of Symbolic Planning with the learning abilities of Deep Reinforcement Learning.

Question Answering reinforcement-learning

Early Fusion for Goal Directed Robotic Vision

no code implementations21 Nov 2018 Aaron Walsman, Yonatan Bisk, Saadia Gabriel, Dipendra Misra, Yoav Artzi, Yejin Choi, Dieter Fox

Building perceptual systems for robotics which perform well under tight computational budgets requires novel architectures which rethink the traditional computer vision pipeline.

Imitation Learning

RMPflow: A Computational Graph for Automatic Motion Policy Generation

1 code implementation16 Nov 2018 Ching-An Cheng, Mustafa Mukadam, Jan Issac, Stan Birchfield, Dieter Fox, Byron Boots, Nathan Ratliff

We develop a novel policy synthesis algorithm, RMPflow, based on geometrically consistent transformations of Riemannian Motion Policies (RMPs).

Robotics Systems and Control

Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience

no code implementations12 Oct 2018 Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, Dieter Fox

In doing so, we are able to change the distribution of simulations to improve the policy transfer by matching the policy behavior in simulation and the real world.

GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning

no code implementations12 Oct 2018 Jacky Liang, Viktor Makoviychuk, Ankur Handa, Nuttapong Chentanez, Miles Macklin, Dieter Fox

Most Deep Reinforcement Learning (Deep RL) algorithms require a prohibitively large number of training samples for learning complex tasks.

Robotics

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

7 code implementations27 Sep 2018 Jonathan Tremblay, Thang To, Balakumar Sundaralingam, Yu Xiang, Dieter Fox, Stan Birchfield

Using synthetic data generated in this manner, we introduce a one-shot deep neural network that is able to perform competitively against a state-of-the-art network trained on a combination of real and synthetic data.

Robotics

SPNets: Differentiable Fluid Dynamics for Deep Neural Networks

1 code implementation15 Jun 2018 Connor Schenck, Dieter Fox

In this paper we introduce Smooth Particle Networks (SPNets), a framework for integrating fluid dynamics with deep networks.

Robotics

The Limits and Potentials of Deep Learning for Robotics

no code implementations18 Apr 2018 Niko Sünderhauf, Oliver Brock, Walter Scheirer, Raia Hadsell, Dieter Fox, Jürgen Leitner, Ben Upcroft, Pieter Abbeel, Wolfram Burgard, Michael Milford, Peter Corke

In this paper we discuss a number of robotics-specific learning, reasoning, and embodiment challenges for deep learning.

Robotics

DeepIM: Deep Iterative Matching for 6D Pose Estimation

1 code implementation ECCV 2018 Yi Li, Gu Wang, Xiangyang Ji, Yu Xiang, Dieter Fox

Estimating the 6D pose of objects from images is an important problem in various applications such as robot manipulation and virtual reality.

6D Pose Estimation 6D Pose Estimation using RGB

Dynamic High Resolution Deformable Articulated Tracking

no code implementations21 Nov 2017 Aaron Walsman, Weilin Wan, Tanner Schmidt, Dieter Fox

The last several years have seen significant progress in using depth cameras for tracking articulated objects such as human bodies, hands, and robotic manipulators.

Pose Estimation

Simulating Action Dynamics with Neural Process Networks

no code implementations ICLR 2018 Antoine Bosselut, Omer Levy, Ari Holtzman, Corin Ennis, Dieter Fox, Yejin Choi

Understanding procedural language requires anticipating the causal effects of actions, even when they are not explicitly stated.

PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

10 code implementations1 Nov 2017 Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, Dieter Fox

We conduct extensive experiments on our YCB-Video dataset and the OccludedLINEMOD dataset to show that PoseCNN is highly robust to occlusions, can handle symmetric objects, and provide accurate pose estimation using only color images as input.

6D Pose Estimation 6D Pose Estimation using RGB +1

SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control

no code implementations2 Oct 2017 Arunkumar Byravan, Felix Leeb, Franziska Meier, Dieter Fox

In this work, we present an approach to deep visuomotor control using structured deep dynamics models.

Visual Semantic Planning using Deep Successor Representations

no code implementations ICCV 2017 Yuke Zhu, Daniel Gordon, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, Ali Farhadi

A crucial capability of real-world intelligent agents is their ability to plan a sequence of actions to achieve their goals in the visual world.

Imitation Learning reinforcement-learning

Re3 : Real-Time Recurrent Regression Networks for Visual Tracking of Generic Objects

10 code implementations17 May 2017 Daniel Gordon, Ali Farhadi, Dieter Fox

Robust object tracking requires knowledge and understanding of the object being tracked: its appearance, its motion, and how it changes over time.

Object Tracking Visual Tracking

DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks

1 code implementation9 Mar 2017 Yu Xiang, Dieter Fox

3D scene understanding is important for robots to interact with the 3D world in a meaningful way.

Scene Understanding

Perceiving and Reasoning About Liquids Using Fully Convolutional Networks

no code implementations5 Mar 2017 Conor Schenck, Dieter Fox

That is, a robot asks the questions What in the visual data stream is liquid?

Reasoning About Liquids via Closed-Loop Simulation

no code implementations5 Mar 2017 Connor Schenck, Dieter Fox

In this paper, we show how to close the loop between liquid simulation and real-time perception.

See the Glass Half Full: Reasoning about Liquid Containers, their Volume and Content

no code implementations ICCV 2017 Roozbeh Mottaghi, Connor Schenck, Dieter Fox, Ali Farhadi

Doing so requires estimating the volume of the cup, approximating the amount of water in the pitcher, and predicting the behavior of water when we tilt the pitcher.

Visual Closed-Loop Control for Pouring Liquids

no code implementations9 Oct 2016 Connor Schenck, Dieter Fox

We propose both a model-based and a model-free method utilizing deep learning for estimating the volume of liquid in a container.

Towards Learning to Perceive and Reason About Liquids

no code implementations2 Aug 2016 Connor Schenck, Dieter Fox

Recent advances in AI and robotics have claimed many incredible results with deep learning, yet no work to date has applied deep learning to the problem of liquid perception and reasoning.

Frame

Detection and Tracking of Liquids with Fully Convolutional Networks

no code implementations20 Jun 2016 Connor Schenck, Dieter Fox

In this paper, we apply fully-convolutional deep neural networks to the tasks of detecting and tracking liquids.

Frame Semantic Segmentation

SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks

no code implementations8 Jun 2016 Arunkumar Byravan, Dieter Fox

We introduce SE3-Nets, which are deep neural networks designed to model and learn rigid body motion from raw point cloud data.

DynamicFusion: Reconstruction and Tracking of Non-Rigid Scenes in Real-Time

no code implementations CVPR 2015 Richard A. Newcombe, Dieter Fox, Steven M. Seitz

We present the first dense SLAM system capable of reconstructing non-rigidly deforming scenes in real-time, by fusing together RGBD scans captured from commodity sensors.

Frame

Building Hierarchies of Concepts via Crowdsourcing

no code implementations27 Apr 2015 Yuyin Sun, Adish Singla, Dieter Fox, Andreas Krause

Hierarchies of concepts are useful in many applications from navigation to organization of objects.

Gaussian Processes for Data-Efficient Learning in Robotics and Control

1 code implementation10 Feb 2015 Marc Peter Deisenroth, Dieter Fox, Carl Edward Rasmussen

Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven learning allows to reduce the amount of engineering knowledge, which is otherwise required.

Gaussian Processes

Multi-Task Policy Search

no code implementations2 Jul 2013 Marc Peter Deisenroth, Peter Englert, Jan Peters, Dieter Fox

Learning policies that generalize across multiple tasks is an important and challenging research topic in reinforcement learning and robotics.

Imitation Learning reinforcement-learning

Multipath Sparse Coding Using Hierarchical Matching Pursuit

no code implementations CVPR 2013 Liefeng Bo, Xiaofeng Ren, Dieter Fox

Complex real-world signals, such as images, contain discriminative structures that differ in many aspects including scale, invariance, and data channel.

Image Classification

Kernel Descriptors for Visual Recognition

no code implementations NeurIPS 2010 Liefeng Bo, Xiaofeng Ren, Dieter Fox

We highlight the kernel view of orientation histograms, and show that they are equivalent to a certain type of match kernels over image patches.

Image Classification Scene Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.