Search Results for author:
Found 19 papers, 9 papers with code
Multimodal foundation world models for generalist embodied agents
Furthermore, by introducing a data-free RL strategy, it lays the groundwork for foundation model-based RL for generalist embodied agents.
RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content
We run a large-scale benchmark comprising several state-of-the-art LLMs to uncover differences in performance across models of various types and sizes in a context-conditional language modeling setting.
VCR: Visual Caption Restoration
This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images.
Capture the Flag: Uncovering Data Insights with Large Language Models
The extraction of a small number of relevant insights from vast amounts of data is a crucial component of data-driven decision-making.
Efficient Dynamics Modeling in Interactive Environments with Koopman Theory
We approach this problem from the lens of Koopman theory, where the nonlinear dynamics of the environment can be linearized in a high-dimensional latent space.
Choreographer: Learning and Adapting Skills in Imagination
Unsupervised skill learning aims to learn a rich repertoire of behaviors without external supervision, providing artificial agents with the ability to control and influence the environment.
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
In this work, we study the URLB and propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent, and a task-aware fine-tuning strategy combined with a new proposed hybrid planner, Dyna-MPC, to adapt the agent for downstream tasks.
Multi-label Iterated Learning for Image Classification with Label Ambiguity
We also show that MILe is effective reducing label noise, achieving state-of-the-art performance on real-world large-scale noisy data such as WebVision.
Ranked #6 on
Image Classification
on WebVision-1000
Touch-based Curiosity for Sparse-Reward Tasks
Robots in many real-world settings have access to force/torque sensors in their gripper and tactile sensing is often necessary in tasks that involve contact-rich motion.
Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation
We propose Pix2Shape, an approach to solve this problem with four components: (i) an encoder that infers the latent 3D representation from an image, (ii) a decoder that generates an explicit 2. 5D surfel-based reconstruction of a scene from the latent code (iii) a differentiable renderer that synthesizes a 2D image from the surfel representation, and (iv) a critic network trained to discriminate between images generated by the decoder-renderer and those from a training distribution.
Adversarial Computation of Optimal Transport Maps
We show that during training, our generator follows the $W_2$-geodesic between the initial and the target distributions.
Pix2Scene: Learning Implicit 3D Representations from Images
Modelling 3D scenes from 2D images is a long-standing problem in computer vision with implications in, e. g., simulation and robotics.
W2GAN: RECOVERING AN OPTIMAL TRANSPORT MAP WITH A GAN
In this work we address the converse question: is it possible to recover an optimal map in a GAN fashion?
Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
Learning inter-domain mappings from unpaired data can improve performance in structured prediction tasks, such as image segmentation, by reducing the need for paired data.
Hierarchical Adversarially Learned Inference
We propose a novel hierarchical generative model with a simple Markovian structure and a corresponding inference model.
A Deep Reinforcement Learning Chatbot (Short Version)
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition.
MINE: Mutual Information Neural Estimation
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks.
Adversarial Generation of Natural Language
Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation.
A hypothesize-and-verify framework for Text Recognition using Deep Recurrent Neural Networks
This paper proposes a hybrid text recognizer using a deep recurrent neural network with multiple layers of abstraction and long range context along with a language model to verify the performance of the deep neural network.
