Search Results for author:
Found 49 papers, 19 papers with code
Learning Attentive Meta-Transfer
Meta-transfer learning seeks to improve the efficiency of learning a new task via both meta-learning and transfer-learning in a setting with a stream of evolving tasks.
Learning and Simulation in Generative Structured World Models
The G-SWM not only unifies the key properties of previous models in a principled framework but also achieves two crucial new abilities, multi-modal uncertainty and situated behavior.
Extendable Long-Horizon Planning via Hierarchical Multiscale Diffusion
This paper tackles a novel problem, extendable long-horizon planning-enabling agents to plan trajectories longer than those in training data without compounding errors.
Monte Carlo Tree Diffusion for System 2 Planning
In this paper, we introduce Monte Carlo Tree Diffusion (MCTD), a novel framework that integrates the generative strength of diffusion models with the adaptive search capabilities of MCTS.
Dreamweaver: Learning Compositional World Representations from Pixels
Furthermore, we show how the modularized concept representations of our model enable compositional imagination, allowing the generation of novel videos by recombining attributes from different objects.
MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memory
In this paper, we argue that the primary cause of failure in many low-level controllers is the absence of an episodic memory system.
Slot State Space Models
Unlike conventional SSMs that maintain a monolithic state vector, SlotSSMs maintains the state as a collection of multiple vectors called slots.
PlanDQ: Hierarchical Plan Orchestration via D-Conductor and Q-Performer
At the low level, we used a Q-learning based approach called the Q-Performer to accomplish these sub-goals.
Learning to Compose: Improving Object Centric Learning by Injecting Compositionality
Learning compositional representation is a key aspect of object-centric learning as it enables flexible systematic generalization and supports complex visual reasoning.
Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming
The proposed agent realizes a version of divide-and-conquer-like strategy in dreaming.
Parallelized Spatiotemporal Binding
While modern best practices advocate for scalable architectures that support long-range interactions, object-centric models are yet to fully embrace these architectures.
Spatially-Aware Transformer for Embodied Agents
Adopting this approach, we demonstrate that memory utilization efficiency can be improved, leading to enhanced accuracy in various place-centric downstream tasks.
Neural Language of Thought Models
The Language of Thought Hypothesis suggests that human cognition operates on a structured, language-like system of mental representations.
Simple Hierarchical Planning with Diffusion
Diffusion-based generative methods have proven effective in modeling trajectories with offline datasets.
Object-Centric Slot Diffusion
The recent success of transformer-based image generative models in object-centric learning highlights the importance of powerful image generators for handling complex scenes.
An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learning
In this paper, we investigate the effectiveness of OCR pre-training for image-based reinforcement learning via empirical experiments.
Neural Systematic Binder
While token-like structured knowledge representations are naturally provided in text, it is elusive how to obtain them for unstructured modalities such as scene images.
Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos
Unsupervised object-centric learning aims to represent the modular, compositional, and causal structure of a scene as a set of object representations and thereby promises to resolve many critical limitations of traditional single-vector representations such as poor systematic generalization.
TransDreamer: Reinforcement Learning with Transformer World Models
We then share this world model with a transformer-based policy network and obtain stability in training a transformer-based RL agent.
Model-based Reinforcement Learning
reinforcement-learning
+2
DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations
Top-performing Model-Based Reinforcement Learning (MBRL) agents, such as Dreamer, learn the world model by reconstructing the image observations.
Model-based Reinforcement Learning
reinforcement-learning
+2
Illiterate DALL-E Learns to Compose
In this paper, we propose a simple but novel slot-based autoencoding architecture, called SLATE, for combining the best of both worlds: learning object-centric representations that allows systematic generalization in zero-shot image generation without text.
Illiterate DALL$\cdot$E Learns to Compose
In experiments, we show that this simple architecture achieves zero-shot generation of novel images without text and better quality in generation than the models based on mixture decoders.
Generative Video Transformer: Can Objects be the Words?
We compare our model with previous RNN-based approaches as well as other possible video transformer baselines.
Structured World Belief for Reinforcement Learning in POMDP
In this paper, we propose Structured World Belief, a model for learning and inference of object-centric belief states.
Generative Scene Graph Networks
We formulate GSGN as a variational autoencoder in which the latent representation is a tree-structured probabilistic scene graph.
Generative Neurosymbolic Machines
In this paper, we propose Generative Neurosymbolic Machines, a generative model that combines the benefits of distributed and symbolic representations to support both structured representations of symbolic components and density-based generation.
Improving Generative Imagination in Object-Centric World Models
Third, a few key abilities for more faithful temporal imagination such as multimodal uncertainty and situation-awareness are missing.
Robustifying Sequential Neural Processes
When tasks change over time, meta-transfer learning seeks to improve the efficiency of learning a new task via both meta-learning and transfer-learning.
ROOTS: Object-Centric Representation and Rendering of 3D Scenes
A crucial ability of human intelligence is to build up models of individual 3D objects from partial scene observations.
SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
Previous approaches for unsupervised object-oriented scene representation learning are either based on spatial-attention or scene-mixture approaches and limited in scalability which is a main obstacle towards modeling real-world scenes.
Generative Hierarchical Models for Parts, Objects, and Scenes
Compositional structures between parts and objects are inherent in natural scenes.
SCALOR: Generative World Models with Scalable Object Representations
Scalability in terms of object density in a scene is a primary challenge in unsupervised sequential object-oriented representation learning.
Neural Multisensory Scene Inference
For embodied agents to infer representations of the underlying 3D physical world they inhabit, they should efficiently combine multisensory cues from numerous trials, e. g., by looking at and touching objects.
Variational Temporal Abstraction
We introduce a variational approach to learning and inference of temporally hierarchical structure and representation for sequential data.
OBJECT-ORIENTED REPRESENTATION OF 3D SCENES
In this paper, we propose a generative model, called ROOTS (Representation of Object-Oriented Three-dimension Scenes), for unsupervised object-wise 3D-scene decomposition and and rendering.
Attentive Sequential Neural Processes
In this paper, we propose the Attentive Sequential Neural Processes (ASNP) that resolve the underfitting in SNP by introducing a novel imaginary context as a latent variable and by applying attention over the imaginary context.
Sequential Neural Processes
In this paper, we propose Sequential Neural Processes (SNP) which incorporates a temporal state-transition model of stochastic processes and thus extends its modeling capabilities to dynamic stochastic processes.
Reinforced Imitation Learning from Observations
Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse.
Reinforced Imitation in Heterogeneous Action Space
Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse.
Domain-Adaptive Single-View 3D Reconstruction
In this paper, we propose a framework to improve over these challenges using adversarial training.
Bayesian Model-Agnostic Meta-Learning
Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem.
Hierarchical Multiscale Recurrent Neural Networks
Multiscale recurrent neural networks have been considered as a promising approach to resolve this issue, yet there has been a lack of empirical evidence showing that this type of models can actually capture the temporal dependencies by discovering the latent hierarchical structure of the sequence.
Ranked #19 on
Language Modelling
on Text8
A Neural Knowledge Language Model
Current language models have a significant limitation in the ability to encode and decode factual knowledge.
Hierarchical Memory Networks
In this paper, we explore a form of hierarchical memory network, which can be considered as a hybrid between hard and soft attention memory networks.
Pointing the Unknown Words
At each time-step, the decision of which softmax layer to use choose adaptively made by an MLP which is conditioned on the context.~We motivate our work from a psychological evidence that humans naturally have a tendency to point towards objects in the context or the environment when the name of an object is not known.~We observe improvements on two tasks, neural machine translation on the Europarl English to French parallel corpora and text summarization on the Gigaword dataset using our proposed model.
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Over the past decade, large-scale supervised learning corpora have enabled machine learning researchers to make substantial advances.
Denoising Criterion for Variational Auto-Encoding Framework
Denoising autoencoders (DAE) are trained to reconstruct their clean inputs with noise injected at the input level, while variational autoencoders (VAE) are trained with noise injected in their stochastic hidden layer, with a regularizer that encourages this noise injection.
Scalable MCMC for Mixed Membership Stochastic Blockmodels
We propose a stochastic gradient Markov chain Monte Carlo (SG-MCMC) algorithm for scalable inference in mixed-membership stochastic blockmodels (MMSB).
Large-Scale Distributed Bayesian Matrix Factorization using Stochastic Gradient MCMC
Despite having various attractive qualities such as high prediction accuracy and the ability to quantify uncertainty and avoid over-fitting, Bayesian Matrix Factorization has not been widely adopted because of the prohibitive cost of inference.
