Search Results for author:
Found 15 papers, 7 papers with code
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
Photorealistic Video Generation with Diffusion Models
We present W. A. L. T, a transformer-based approach for photorealistic video generation via diffusion modeling.
Ranked #1 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Holistic Evaluation of Text-To-Image Models
The stunning qualitative improvement of recent text-to-image models has led to their widespread attention and adoption.
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation.
Ranked #2 on
Video Prediction
on Kinetics-600 12 frames, 64x64
RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
With RoboCat, we demonstrate the ability to generalise to new tasks and robots, both zero-shot as well as through adaptation using only 100-1000 examples for the target task.
GreenMO: Virtualized User-proportionate MIMO
The reason is that traditionally MIMO requires a separate RF chain per antenna, so the power consumption scales with number of antennas, instead of number of users, hence becomes energy inefficient.
VIMA: General Robot Manipulation with Multimodal Prompts
We show that a wide spectrum of robot manipulation tasks can be expressed with multimodal prompts, interleaving textual and visual tokens.
MaskViT: Masked Visual Pre-Training for Video Prediction
This work shows that we can create good video prediction models by pre-training transformers via masked visual modeling.
MetaMorph: Learning Universal Controllers with Transformers
Multiple domains like vision, natural language, and audio are witnessing tremendous progress by leveraging Transformers for large scale pre-training followed by task specific fine tuning.
Embodied Intelligence via Learning and Evolution
However, the principles governing relations between environmental complexity, evolved morphology, and the learnability of intelligent control, remain elusive, partially due to the substantial challenge of performing large-scale in silico experiments on evolution and learning.
WiForce: Wireless Sensing and Localization of Contact Forces on a Space Continuum
Contact force is a natural way for humans to interact with the physical world around us.
LVIS: A Dataset for Large Vocabulary Instance Segmentation
We plan to collect ~2 million high-quality instance segmentation masks for over 1000 entry-level object categories in 164k images.
Image Generation from Scene Graphs
To overcome this limitation we propose a method for generating images from scene graphs, enabling explicitly reasoning about objects and their relationships.
Ranked #4 on
Layout-to-Image Generation
on Visual Genome 64x64
Image Generation from Scene Graphs
Layout-to-Image Generation
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Understanding human motion behavior is critical for autonomous moving platforms (like self-driving cars and social robots) if they are to navigate human-centric environments.
Ranked #4 on
Trajectory Prediction
on ETH
Characterizing and Improving Stability in Neural Style Transfer
Recent progress in style transfer on images has focused on improving the quality of stylized images and speed of methods.
