Search Results for author:
Found 14 papers, 4 papers with code
From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons
We examine the capability of Multimodal Large Language Models (MLLMs) to tackle diverse domains that extend beyond the traditional language and vision tasks these models are typically trained on.
On the Modeling Capabilities of Large Language Models for Sequential Decision Making
Large pretrained models are showing increasingly better performance in reasoning and planning tasks across different modalities, opening the possibility to leverage them for complex sequential decision making problems.
Grounding Multimodal Large Language Models in Actions
For discrete actions, we demonstrate that semantically aligning these actions with the native output token space of the MLLM leads to the strongest performance.
Poly-View Contrastive Learning
Contrastive learning typically matches pairs of related views among a number of unrelated negative views.
Large Language Models as Generalizable Policies for Embodied Tasks
We show that large language models (LLMs) can be adapted to be generalizable policies for embodied visual tasks.
Value function estimation using conditional diffusion models for control
A fairly reliable trend in deep reinforcement learning is that the performance scales with the number of parameters, provided a complimentary scaling in amount of training data.
Test Sample Accuracy Scales with Training Sample Density in Neural Networks
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in representation space.
Pretraining Representations for Data-Efficient Reinforcement Learning
Data efficiency is a key challenge for deep reinforcement learning.
Ranked #3 on
Atari Games 100k
on Atari 100k
(using extra training data)
Cross-Modal Information Maximization for Medical Imaging: CMIM
In hospitals, data are siloed to specific information systems that make the same information available under different modalities such as the different medical imaging exams the patient undergoes (CT scans, MRI, PET, Ultrasound, etc.)
Locality and compositionality in zero-shot learning
In this work we study locality and compositionality in the context of learning representations for Zero Shot Learning (ZSL).
Tell, Draw, and Repeat: Generating and Modifying Images Based on Continual Linguistic Instruction
Conditional text-to-image generation is an active area of research, with many possible applications.
Ranked #2 on
Text-to-Image Generation
on GeNeVA (i-CLEVR)
Mutual Information Neural Estimation
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks.
Learning Generative Models with Locally Disentangled Latent Factors
One of the most successful techniques in generative models has been decomposing a complicated generation task into a series of simpler generation tasks.
GibbsNet: Iterative Adversarial Inference for Deep Graphical Models
Directed latent variable models that formulate the joint distribution as $p(x, z) = p(z) p(x \mid z)$ have the advantage of fast and exact sampling.
