Search Results for author: Danny Driess

Found 16 papers, 4 papers with code

Large Language Models as General Pattern Machines

no code implementations10 Jul 2023 Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng

We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art.

In-Context Learning

Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents

no code implementations NeurIPS 2023 Wenlong Huang, Fei Xia, Dhruv Shah, Danny Driess, Andy Zeng, Yao Lu, Pete Florence, Igor Mordatch, Sergey Levine, Karol Hausman, Brian Ichter

Recent progress in large language models (LLMs) has demonstrated the ability to learn and leverage Internet-scale knowledge through pre-training with autoregressive models.

Language Modelling Text Generation

Reinforcement Learning with Neural Radiance Fields

no code implementations3 Jun 2022 Danny Driess, Ingmar Schubert, Pete Florence, Yunzhu Li, Marc Toussaint

This paper demonstrates that learning state representations with supervision from Neural Radiance Fields (NeRFs) can improve the performance of RL compared to other learned representations or even low-dimensional, hand-engineered state information.

reinforcement-learning Reinforcement Learning (RL)

Learning Multi-Object Dynamics with Compositional Neural Radiance Fields

no code implementations24 Feb 2022 Danny Driess, Zhiao Huang, Yunzhu Li, Russ Tedrake, Marc Toussaint

We present a method to learn compositional multi-object dynamics models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural networks.


Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning

no code implementations2 Oct 2021 Danny Driess, Jung-Su Ha, Marc Toussaint, Russ Tedrake

We show that representing objects as signed-distance fields not only enables to learn and represent a variety of models with higher accuracy compared to point-cloud and occupancy measure representations, but also that SDF-based models are suitable for optimization-based planning.

Learning Neural Implicit Functions as Object Representations for Robotic Manipulation

no code implementations29 Sep 2021 Jung-Su Ha, Danny Driess, Marc Toussaint

Robotic manipulation planning is the problem of finding a sequence of robot configurations that involves interactions with objects in the scene, e. g., grasp, placement, tool-use, etc.

Open-Ended Question Answering Robot Manipulation

Deep 6-DoF Tracking of Unknown Objects for Reactive Grasping

no code implementations9 Mar 2021 Marc Tuscher, Julian Hörz, Danny Driess, Marc Toussaint

We propose a robotic manipulation system, which is able to grasp a wide variety of formerly unseen objects and is robust against object perturbations and inferior grasping points.

Object Object Tracking +1

Describing Physics For Physical Reasoning: Force-based Sequential Manipulation Planning

1 code implementation28 Feb 2020 Marc Toussaint, Jung-Su Ha, Danny Driess

Physical reasoning is a core aspect of intelligence in animals and humans.


Cannot find the paper you are looking for? You can Submit a new open access paper.