Search Results for author:
Found 138 papers, 58 papers with code
Learning to Build by Building Your Own Instructions
By disassembling an unseen assembly and periodically saving images of it, the agent is able to create a set of instructions so that it has the information necessary to rebuild it.
OptiGrasp: Optimized Grasp Pose Detection Using RGB Images for Warehouse Picking Robots
In warehouse environments, robots require robust picking capabilities to manage a wide variety of objects.
PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks
To kickstart the benchmark, we extended several state-of-the art methods to bimanual manipulation and also present a language-conditioned behavioral cloning agent -- PerAct2, which enables the learning and execution of bimanual 6-DoF manipulation tasks.
Manipulate-Anything: Automating Real-World Robots using Vision-Language Models
Large-scale endeavors like and widespread community efforts such as Open-X-Embodiment have contributed to growing the scale of robot demonstration data.
3D-MVP: 3D Multiview Pretraining for Robotic Manipulation
Our results suggest that 3D-aware pretraining is a promising approach to improve sample efficiency and generalization of vision-based robotic manipulation policies.
RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
In spite of the recent adoption of vision language models (VLMs) to control robot behavior, VLMs struggle to precisely articulate robot actions using language.
RVT-2: Learning Precise Manipulation from Few Demonstrations
In this work, we study how to build a robotic system that can solve multiple 3D manipulation tasks given language instructions.
Ranked #3 on
Robot Manipulation Generalization
on GEMBench
URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images
We present an integrated end-to-end pipeline that generates simulation scenes complete with articulated kinematic and dynamic structures from real-world images and use these for training robotic control policies.
IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning
Imitation learning is a promising paradigm for training robot control policies, but these policies can suffer from distribution shift, where the conditions at evaluation time differ from those in the training data.
ASID: Active Exploration for System Identification in Robotic Manipulation
In this work, we propose a learning system that can leverage a small amount of real-world data to autonomously refine a simulation model and then plan an accurate control strategy that can be deployed in the real world.
AdaDemo: Data-Efficient Demonstration Expansion for Generalist Robotic Agent
Encouraged by the remarkable achievements of language and vision foundation models, developing generalist robotic agents through imitation learning, using large demonstration datasets, has become a prominent area of interest in robot learning.
Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects
We address the problem of building digital twins of unknown articulated objects from two RGBD scans of the object at different articulation states.
THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
To realize effective large-scale, real-world robotic applications, we must evaluate how well our robot policies adapt to changes in environmental conditions.
Ranked #1 on
Robot Manipulation Generalization
on The COLOSSEUM
STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots
Segmentation and tracking of unseen object instances in discrete frames pose a significant challenge in dynamic industrial robotic contexts, such as distribution warehouses.
M2T2: Multi-Task Masked Transformer for Object-centric Pick and Place
With the advent of large language models and large-scale robotic datasets, there has been tremendous progress in high-level decision-making for object manipulation.
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Imitation learning from a large set of human demonstrations has proved to be an effective paradigm for building capable robot agents.
Human-in-the-Loop Task and Motion Planning for Imitation Learning
Finally, we collected 2. 1K demos with HITL-TAMP across 12 contact-rich, long-horizon tasks and show that the system often produces near-perfect agents.
NEWTON: Are Large Language Models Capable of Physical Reasoning?
To address this gap, we introduce NEWTON, a repository and benchmark for evaluating the physics reasoning skills of LLMs.
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
For real-world experiments, AnyTeleop can outperform a previous system that was designed for a specific robot hardware with a higher success rate, using the same robot.
Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangement
We propose a system for rearranging objects in a scene to achieve a desired object-scene placing relationship, such as a book inserted in an open slot of a bookshelf.
RVT: Robotic View Transformer for 3D Object Manipulation
In simulations, we find that a single RVT model works well across 18 RLBench tasks with 249 task variations, achieving 26% higher relative success than the existing state-of-the-art method (PerAct).
Ranked #6 on
Robot Manipulation
on RLBench
AR2-D2:Training a Robot Without a Robot
By contrast, we introduce AR2-D2: a system for collecting demonstrations which (1) does not require people with specialized training, (2) does not require any real robots during data collection, and therefore, (3) enables manipulation of diverse objects with a real robot.
DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability
Task and Motion Planning (TAMP) approaches are effective at planning long-horizon autonomous robot manipulation.
Imitating Task and Motion Planning with Visuomotor Transformers
In this work, we show that the combination of large-scale datasets generated by TAMP supervisors and flexible Transformer models to fit them is a powerful paradigm for robot manipulation.
CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation
CabiNet is a collision model that accepts object and scene point clouds, captured from a single-view depth observation, and predicts collisions for SE(3) object poses in the scene.
Partial-View Object View Synthesis via Filtered Inversion
At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.
Learning Human-to-Robot Handovers from Point Clouds
We propose the first framework to learn control policies for vision-based human-to-robot handovers, a critical task for human-robot interaction.
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
We present a near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence, while simultaneously performing neural 3D reconstruction of the object.
MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare
Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner.
Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning
The core of our algorithm is PIGINet, a novel Transformer-based learning method that takes in a task plan, the goal, and the initial state, and predicts the probability of finding motion trajectories associated with the task plan.
DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality
Our policies are trained to adapt to a wide range of conditions in simulation.
One-Shot Neural Fields for 3D Object Understanding
At test-time, we build the representation from a single RGB input image observing the scene from only one viewpoint.
Motion Policy Networks
Collision-free motion generation in unknown environments is a core building block for robot manipulation.
DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrations
The policy learned from our dataset can generalize well on unseen object poses in both simulation and the real world
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
To ameliorate that effort, large language models (LLMs) can be used to score potential next actions during task planning, and even generate action sequences directly, given an instruction in natural language with no additional domain information.
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation
With this formulation, we train a single multi-task Transformer for 18 RLBench tasks (with 249 variations) and 7 real-world tasks (with 18 variations) from just a few demonstrations per task.
Ranked #7 on
Robot Manipulation
on RLBench
Break and Make: Interactive Structural Understanding Using LEGO Bricks
We pair this simulator with a new dataset of fan-made LEGO creations that have been uploaded to the internet in order to provide complex scenes containing over a thousand unique brick shapes.
Neural Motion Fields: Encoding Grasp Trajectories as Implicit Value Functions
The pipeline of current robotic pick-and-place methods typically consists of several stages: grasp pose detection, finding inverse kinematic solutions for the detected poses, planning a collision-free trajectory, and then executing the open-loop trajectory to the grasp pose with a low-level tracking controller.
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
We analyze the performance of a set of baselines and show a correlation with a real-world evaluation.
Factory: Fast Contact for Robotic Assembly
We aim for Factory to open the doors to using simulation for robotic assembly, as well as many other contact-rich applications in robotics.
Correcting Robot Plans with Natural Language Feedback
In this paper, we explore natural language as an expressive and flexible tool for robot correction.
Model Predictive Control for Fluid Human-to-Robot Handovers
However, how to responsively generate smooth motions to take an object from a human is still an open question.
DiSECt: A Differentiable Simulator for Parameter Inference and Control in Robotic Cutting
In this work, we present DiSECt: the first differentiable simulator for cutting soft materials.
IFOR: Iterative Flow Minimization for Robotic Object Rearrangement
Accurate object rearrangement from vision is a crucial problem for a wide variety of real-world robotics applications in unstructured environments.
iCaps: Iterative Category-level Object Pose and Shape Estimation
The auto-encoder can be used in a particle filter framework to estimate and track 6D poses of objects in a category.
A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation
Deformable object manipulation remains a challenging task in robotics research.
Learning Perceptual Concepts by Bootstrapping from Human Queries
Second, we treat this low-dimensional concept as an automatic labeler to synthesize a large-scale high-dimensional data set with the simulator.
From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains.
StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects
Geometric organization of objects into semantically meaningful arrangements pervades the built world.
Continuous-Time Fitted Value Iteration for Robust Policies
Especially for continuous control, solving this differential equation and its extension the Hamilton-Jacobi-Isaacs equation, is important as it yields the optimal policy that achieves the maximum reward on a give task.
Generalizing Successor Features to continuous domains for Multi-task Learning
In this paper, we consider the problem of continuous control for various robot manipulation tasks with an explicit representation that promotes skill reuse while learning multiple tasks, related through the reward function.
CLIPort: What and Where Pathways for Robotic Manipulation
We even learn one multi-task policy for 10 simulated and 9 real-world tasks that is better or comparable to single-task policies.
SORNet: Spatial Object-Centric Representations for Sequential Manipulation
Sequential manipulation tasks require a robot to perceive the state of an environment and plan a sequence of actions leading to a desired goal state.
Predicting Stable Configurations for Semantic Placement of Novel Objects
We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution
Natural language provides an accessible and expressive interface to specify long-term tasks for robotic agents.
BayesSimIG: Scalable Parameter Inference for Adaptive Domain Randomization with IsaacGym
This paper outlines BayesSimIG: a library that provides an implementation of BayesSim integrated with the recently released NVIDIA IsaacGym.
LanguageRefer: Spatial-Language Model for 3D Visual Grounding
Specifically, given a reconstructed 3D scene in the form of point clouds with 3D bounding boxes of potential object candidates, and a language utterance referring to a target object in the scene, our model successfully identifies the target object from a set of potential candidates.
RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks
We postulate that a network architecture that encodes relations between objects at a high-level can be beneficial.
NeRP: Neural Rearrangement Planning for Unknown Objects
We propose NeRP (Neural Rearrangement Planning), a deep learning based approach for multi-step neural object rearrangement planning which works with never-before-seen objects, that is trained on simulation data, and generalizes to the real world.
Robust Value Iteration for Continuous Control Tasks
The adversarial perturbations encourage a optimal policy that is robust to changes in the dynamics.
Value Iteration in Continuous Actions, States and Time
This algorithm enables dynamic programming for continuous states and actions with a known dynamics model.
DexYCB: A Benchmark for Capturing Hand Grasping of Objects
We introduce DexYCB, a new dataset for capturing hand grasping of objects.
RGB-D Local Implicit Function for Depth Completion of Transparent Objects
Key to our approach is a local implicit neural representation built on ray-voxel pairs that allows our method to generalize to unseen objects and achieve fast inference speed.
Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes
Our novel grasp representation treats 3D points of the recorded point cloud as potential grasp contacts.
Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees
The policy structure provides the user an interface to 1) specifying the spaces that are directly relevant to the completion of the tasks, and 2) designing policies for certain tasks that do not need to be learned.
Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshop
This report presents the debates, posters, and discussions of the Sim2Real workshop held in conjunction with the 2020 edition of the "Robotics: Science and System" conference.
Object Rearrangement Using Learned Implicit Collision Functions
The learned model outperforms both traditional pipelines and learned ablations by 9. 8% in accuracy on a dataset of simulated collision queries and is 75x faster than the best-performing baseline.
ACRONYM: A Large-Scale Grasp Dataset Based on Simulation
We introduce ACRONYM, a dataset for robot grasp planning based on physics simulation.
Reactive Long Horizon Task Execution via Visual Skill and Precondition Models
Zero-shot execution of unseen robotic tasks is important to allowing robots to perform a wide variety of tasks in human environments, but collecting the amounts of data necessary to train end-to-end policies in the real-world is often infeasible.
Reactive Human-to-Robot Handovers of Arbitrary Objects
We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects, a user study with naive users (N=6) handing over a subset of 15 objects, and a systematic evaluation examining different ways of handing objects.
A User's Guide to Calibrating Robotics Simulators
Simulators are a critical component of modern robotics research.
Stein Variational Model Predictive Control
By casting MPC as a Bayesian inference problem, we employ variational methods for posterior computation, naturally encoding the complexity and multi-modality of the decision making problem.
Multimodal Trajectory Prediction via Topological Invariance for Navigation at Uncontrolled Intersections
Our key insight is that the geometric structure of the intersection and the incentive of agents to move efficiently and avoid collisions (rationality) reduces the space of likely behaviors, effectively relaxing the problem of trajectory prediction.
Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds
We demonstrate that our learned policy can be integrated into a tabletop 6D grasping system and a human-robot handover system to improve the grasping performance of unseen objects.
DeepGMR: Learning Latent Gaussian Mixture Models for Registration
Point cloud registration is a fundamental problem in 3D computer vision, graphics and robotics.
Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation
In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data.
Unseen Object Instance Segmentation for Robotic Environments
We also show that our method can segment unseen objects for robot grasping.
Causal Discovery in Physical Systems from Videos
We assume access to different configurations and environmental conditions, i. e., data from unknown interventions on the underlying system; thus, we can hope to discover the correct underlying causal graph without explicit interventions.
Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems
The complex motions are encoded as rollouts of a stable dynamical system, which, under a change of coordinates defined by a diffeomorphism, is equivalent to a simple, hand-specified dynamical system.
Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning
Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state.
Watching the World Go By: Representation Learning from Unlabeled Videos
Recent single image unsupervised representation learning techniques show remarkable success on a variety of tasks.
Human Grasp Classification for Reactive Human-to-Robot Handovers
In this paper, we propose an approach for human-to-robot handovers in which the robot meets the human halfway, by classifying the human's grasp of the object and quickly planning a trajectory accordingly to take the object from the human's hand according to their intent.
Transferable Task Execution from Pixels through Deep Planning Domain Learning
On the other hand, symbolic planning methods such as STRIPS have long been able to solve new problems given only a domain definition and a symbolic goal, but these approaches often struggle on the real world robotic tasks due to the challenges of grounding these symbols from sensor data in a partially-observable world.
Information Theoretic Model Predictive Q-Learning
Model-free Reinforcement Learning (RL) works well when experience can be collected cheaply and model-based RL is effective when system dynamics can be modeled accurately.
A Billion Ways to Grasp: An Evaluation of Grasp Sampling Schemes on a Dense, Physics-based Grasp Data Set
With the increasing speed and quality of physics simulations, generating large-scale grasping data sets that feed learning algorithms is becoming more and more popular.
6-DOF Grasping for Target-driven Object Manipulation in Clutter
Grasping in cluttered environments is a fundamental but challenging robotic skill.
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks.
LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation
We evaluate the performance of our method for unseen object pose estimation on MOPED as well as the ModelNet and LINEMOD datasets.
Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection
In robot manipulation, planning the motion of a robot manipulator to grasp an object is a fundamental problem.
Robotics
Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies
We propose a Grasping Objects Approach for Tactile (GOAT) robotic hands, learning to overcome the reality gap problem.
Robotics
Camera-to-Robot Pose Estimation from a Single Image
We show experimental results for three different camera sensors, demonstrating that our approach is able to achieve accuracy with a single frame that is better than that of classic off-line hand-eye calibration using multiple frames.
Robotics
Motion Reasoning for Goal-Based Imitation Learning
We further show that by using the automatically inferred goal from the video demonstration, our robot is able to reproduce the same task in a real kitchen environment.
IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data
For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task.
Online Replanning in Belief Space for Partially Observable Task and Motion Problems
To solve multi-step manipulation tasks in the real world, an autonomous robot must take actions to observe its environment and react to unexpected observations.
Motion-Nets: 6D Tracking of Unknown Objects in Unseen Environments using RGB
In this work, we bridge the gap between recent pose estimation and tracking work to develop a powerful method for robots to track objects in their surroundings.
Conditional Driving from Natural Language Instructions
Widespread adoption of self-driving cars will depend not only on their safety but largely on their ability to interact with human users.
Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping
RMPfusion supplements RMPflow with weight functions that can hierarchically reshape the Lyapunov functions of the subtask RMPs according to the current configuration of the robot and environment.
DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System
Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks.
Self-supervised 6D Object Pose Estimation for Robot Manipulation
In this way, our system is able to continuously collect data and improve its pose estimation modules.
Robotics
Part Segmentation for Highly Accurate Deformable Tracking in Occlusions via Fully Convolutional Neural Networks
While recent work has shown direct estimation techniques can be quite powerful, geometric tracking methods using point clouds can provide a very high level of 3D accuracy which is necessary for many robotic applications.
The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
We show that our method, trained on this dataset, can produce sharp and accurate masks, outperforming state-of-the-art methods on unseen object instance segmentation.
BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators
We introduce BayesSim, a framework for robotics simulations allowing a full Bayesian treatment for the parameters of the simulator.
6-DOF GraspNet: Variational Grasp Generation for Object Manipulation
We evaluate our approach in simulation and real-world robot experiments.
PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking
In this work, we formulate the 6D object pose tracking problem in the Rao-Blackwellized particle filtering framework, where the 3D rotation and the 3D translation of an object are decoupled.
ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact
Using a dataset of contact demonstrations from humans grasping diverse household objects, we synthesize functional grasps for three hand models and two functional intents.
Prospection: Interpretable Plans From Language By Predicting the Future
High-level human instructions often correspond to behaviors with multiple implicit steps.
What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planning
In this work we propose Hierarchical Planning and Reinforcement Learning (HIP-RL), a method for merging the benefits and capabilities of Symbolic Planning with the learning abilities of Deep Reinforcement Learning.
Object Discovery in Videos as Foreground Motion Clustering
We consider the problem of providing dense segmentation masks for object discovery in videos.
Early Fusion for Goal Directed Robotic Vision
Building perceptual systems for robotics which perform well under tight computational budgets requires novel architectures which rethink the traditional computer vision pipeline.
RMPflow: A Computational Graph for Automatic Motion Policy Generation
We develop a novel policy synthesis algorithm, RMPflow, based on geometrically consistent transformations of Riemannian Motion Policies (RMPs).
Robotics Systems and Control
GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning
Most Deep Reinforcement Learning (Deep RL) algorithms require a prohibitively large number of training samples for learning complex tasks.
Robotics
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
In doing so, we are able to change the distribution of simulations to improve the policy transfer by matching the policy behavior in simulation and the real world.
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
Using synthetic data generated in this manner, we introduce a one-shot deep neural network that is able to perform competitively against a state-of-the-art network trained on a combination of real and synthetic data.
Robotics
SPNets: Differentiable Fluid Dynamics for Deep Neural Networks
In this paper we introduce Smooth Particle Networks (SPNets), a framework for integrating fluid dynamics with deep networks.
Robotics
The Limits and Potentials of Deep Learning for Robotics
In this paper we discuss a number of robotics-specific learning, reasoning, and embodiment challenges for deep learning.
Robotics
DeepIM: Deep Iterative Matching for 6D Pose Estimation
Estimating the 6D pose of objects from images is an important problem in various applications such as robot manipulation and virtual reality.
Ranked #1 on
6D Pose Estimation using RGB
on YCB-Video
IQA: Visual Question Answering in Interactive Environments
Our experiments show that our proposed model outperforms popular single controller based methods on IQUAD V1.
Dynamic High Resolution Deformable Articulated Tracking
The last several years have seen significant progress in using depth cameras for tracking articulated objects such as human bodies, hands, and robotic manipulators.
Simulating Action Dynamics with Neural Process Networks
Understanding procedural language requires anticipating the causal effects of actions, even when they are not explicitly stated.
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
We conduct extensive experiments on our YCB-Video dataset and the OccludedLINEMOD dataset to show that PoseCNN is highly robust to occlusions, can handle symmetric objects, and provide accurate pose estimation using only color images as input.
Ranked #3 on
6D Pose Estimation using RGB
on YCB-Video
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control
In this work, we present an approach to deep visuomotor control using structured deep dynamics models.
Visual Semantic Planning using Deep Successor Representations
A crucial capability of real-world intelligent agents is their ability to plan a sequence of actions to achieve their goals in the visual world.
Re3 : Real-Time Recurrent Regression Networks for Visual Tracking of Generic Objects
Robust object tracking requires knowledge and understanding of the object being tracked: its appearance, its motion, and how it changes over time.
DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks
3D scene understanding is important for robots to interact with the 3D world in a meaningful way.
Perceiving and Reasoning About Liquids Using Fully Convolutional Networks
That is, a robot asks the questions What in the visual data stream is liquid?
Reasoning About Liquids via Closed-Loop Simulation
In this paper, we show how to close the loop between liquid simulation and real-time perception.
See the Glass Half Full: Reasoning about Liquid Containers, their Volume and Content
Doing so requires estimating the volume of the cup, approximating the amount of water in the pitcher, and predicting the behavior of water when we tilt the pitcher.
Visual Closed-Loop Control for Pouring Liquids
We propose both a model-based and a model-free method utilizing deep learning for estimating the volume of liquid in a container.
Towards Learning to Perceive and Reason About Liquids
Recent advances in AI and robotics have claimed many incredible results with deep learning, yet no work to date has applied deep learning to the problem of liquid perception and reasoning.
Detection and Tracking of Liquids with Fully Convolutional Networks
In this paper, we apply fully-convolutional deep neural networks to the tasks of detecting and tracking liquids.
SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks
We introduce SE3-Nets, which are deep neural networks designed to model and learn rigid body motion from raw point cloud data.
DynamicFusion: Reconstruction and Tracking of Non-Rigid Scenes in Real-Time
We present the first dense SLAM system capable of reconstructing non-rigidly deforming scenes in real-time, by fusing together RGBD scans captured from commodity sensors.
Building Hierarchies of Concepts via Crowdsourcing
Hierarchies of concepts are useful in many applications from navigation to organization of objects.
Gaussian Processes for Data-Efficient Learning in Robotics and Control
Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven learning allows to reduce the amount of engineering knowledge, which is otherwise required.
Multi-Task Policy Search
Learning policies that generalize across multiple tasks is an important and challenging research topic in reinforcement learning and robotics.
Multipath Sparse Coding Using Hierarchical Matching Pursuit
Complex real-world signals, such as images, contain discriminative structures that differ in many aspects including scale, invariance, and data channel.
A Joint Model of Language and Perception for Grounded Attribute Learning
As robots become more ubiquitous and capable, it becomes ever more important to enable untrained users to easily interact with them.
Kernel Descriptors for Visual Recognition
We highlight the kernel view of orientation histograms, and show that they are equivalent to a certain type of match kernels over image patches.
