Search Results for author:
Found 38 papers, 13 papers with code
Compositional Visual Generation with Composable Diffusion Models
Large text-guided diffusion models, such as DALLE-2, are able to generate stunning photorealistic images given natural language descriptions.
Planning with Diffusion for Flexible Behavior Synthesis
Model-based reinforcement learning methods often use learning only for the purpose of estimating an approximate dynamics model, offloading the rest of the decision-making work to classical trajectory optimizers.
Streaming Inference for Infinite Non-Stationary Clustering
Learning from a continuous stream of non-stationary data in an unsupervised manner is arguably one of the most common and most challenging settings facing intelligent agents.
Learning Neural Acoustic Fields
By modeling acoustic propagation in a scene as a linear time-invariant system, NAFs learn to continuously map all emitter and listener location pairs to a neural impulse response function that can then be applied to arbitrary sounds.
Kubric: A scalable dataset generator
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details.
Pre-Trained Language Models for Interactive Decision-Making
The agent iteratively learns by interacting with the environment, relabeling the language goal of past 'failed' experiences, and updating the policy in a self-supervised loop.
Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Our performance generalizes across both object instances and 6-DoF object poses, and significantly outperforms a recent baseline that relies on 2D descriptors.
Learning to Compose Visual Relations
The visual world around us can be described as a structured set of objects and their associated relations.
Learning Signal-Agnostic Manifolds of Neural Fields
We leverage neural fields to capture the underlying structure in image, shape, audio and cross-modal audiovisual domains in a modality-independent manner.
Unsupervised Learning of Compositional Energy Concepts
In this work, we propose COMET, which discovers and represents concepts as separate energy functions, enabling us to represent both global concepts as well as objects under a unified framework.
The Neural MMO Platform for Massively Multiagent Research
Neural MMO is a computationally accessible research platform that combines large agent populations, long time horizons, open-ended tasks, and modular game systems.
Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions
Our task poses unique challenges as a system does not know what types of human-object interactions are present in a video or the actual spatiotemporal location of the human and the object.
Language Model Pre-training Improves Generalization in Policy Learning
Additional experiments explore the role of language-based encodings in these results; we find that it is possible to train a simple adapter layer that maps from observations and action histories to LM embeddings, and thus that language modeling provides an effective initializer even for tasks with no language as input or output.
Curious Representation Learning for Embodied Intelligence
Instead, it must explore its environment to acquire the data it will learn from.
3D Shape Generation and Completion through Point-Voxel Diffusion
We propose a novel approach for probabilistic generative modeling of 3D shapes.
Unsupervised Discovery of 3D Physical Objects
We study the problem of unsupervised physical object discovery.
Neural Radiance Flow for 4D View Synthesis and Video Processing
We present a method, Neural Radiance Flow (NeRFlow), to learn a 4D spatial-temporal representation of a dynamic scene from a set of RGB images.
Improved Contrastive Divergence Training of Energy Based Models
Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability.
Compositional Visual Generation with Energy Based Models
A vital aspect of human intelligence is the ability to compose increasingly complex concepts out of simpler ideas, enabling both rapid learning and adaptation of knowledge.
Energy-Based Models for Continual Learning
We motivate Energy-Based Models (EBMs) as a promising model class for continual learning problems.
A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects
We present a framework for solving long-horizon planning problems involving manipulation of rigid objects that operates directly from a point-cloud observation, i. e. without prior object models.
Learning Object-Based State Estimators for Household Robots
The robot may be called upon later to retrieve objects and will need a long-term object-based memory in order to know how to find them.
Learning Online Data Association
When an agent interacts with a complex environment, it receives a stream of percepts in which it may detect entities, such as objects or people.
Unsupervised Discovery of 3D Physical Objects from Video
We study the problem of unsupervised physical object discovery.
Energy-based models for atomic-resolution protein conformations
We propose an energy-based model (EBM) of protein conformations that operates at atomic scale.
Compositional Visual Generation and Inference with Energy Based Models
A vital aspect of human intelligence is the ability to compose increasingly complex concepts out of simpler ideas, enabling both rapid learning and adaptation of knowledge.
Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks
We present Neural MMO, a massively multiagent game environment inspired by MMOs and discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO.
Observational Overfitting in Reinforcement Learning
A major component of overfitting in model-free reinforcement learning (RL) involves the case where the agent may mistakenly correlate reward with certain spurious features from the observations generated by the Markov Decision Process (MDP).
Implicit Generation and Modeling with Energy Based Models
Energy based models (EBMs) are appealing due to their generality and simplicity in likelihood modeling, but have been traditionally difficult to train.
Ranked #3 on
Image Generation
on Stacked MNIST
Learning Good Policies By Learning Good Perceptual Models
Reinforcement learning (RL) has led to increasingly complex looking behavior in recent years.
Model Based Planning with Energy Based Models
We provide an online algorithm to train EBMs while interacting with the environment, and show that EBMs allow for significantly better online learning than corresponding feed-forward networks.
An Empirical Study on Hyperparameters and their Interdependence for RL Generalization
Recent results in Reinforcement Learning (RL) have shown that agents with limited training environments are susceptible to a large amount of overfitting across many domains.
Task-Agnostic Dynamics Priors for Deep Reinforcement Learning
While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is often challenging and requires substantial interaction with the environment.
Neural MMO: A massively multiplayer game environment for intelligent agents
We demonstrate how this platform can be used to study behavior and learning in large populations of neural agents.
Implicit Generation and Generalization in Energy-Based Models
Energy based models (EBMs) are appealing due to their generality and simplicity in likelihood modeling, but have been traditionally difficult to train.
Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents
The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources.
Learning to Exploit Stability for 3D Scene Parsing
We first show that applying physics supervision to an existing scene understanding model increases performance, produces more stable predictions, and allows training to an equivalent performance level with fewer annotated training examples.
Learning Physics Priors for Deep Reinforcement Learing
While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is challenging and often requires substantial interactions with the environment.
