Search Results for author:
Found 12 papers, 3 papers with code
No RL, No Simulation: Learning to Navigate without Navigating
Most prior methods for learning navigation policies require access to simulation environments, as they need online policy interaction and rely on ground-truth maps for rewards.
Where Are You? Localization from Embodied Dialog
In this paper, we focus on the LED task -- providing a strong baseline model with detailed ablations characterizing both dataset biases and the importance of various modeling choices.
Text and Click inputs for unambiguous open vocabulary instance segmentation
We propose a new segmentation process, Text + Click segmentation, where a model takes as input an image, a text phrase describing a class to segment, and a single foreground click specifying the instance to segment.
Deep Tracking: Visual Tracking Using Deep Convolutional Networks
In this paper, we study a discriminatively trained deep convolutional network for the task of visual tracking.
Learning to Localize and Align Fine-Grained Actions to Sparse Instructions
Automatic generation of textual video descriptions that are time-aligned with video content is a long-standing goal in computer vision.
Action2Vec: A Crossmodal Embedding Approach to Action Learning
We describe a novel cross-modal embedding space for actions, named Action2Vec, which combines linguistic cues from class labels with spatio-temporal features derived from video clips.
Tripping through time: Efficient Localization of Activities in Videos
Localizing moments in untrimmed videos via language queries is a new and interesting task that requires the ability to accurately ground language into video.
Transformer-based Localization from Embodied Dialog with Large-scale Pre-training
We address the challenging task of Localization via Embodied Dialog (LED).
Learning a Visually Grounded Memory Assistant
We introduce a novel interface for large scale collection of human memory and assistance.
Which way is `right'?: Uncovering limitations of Vision-and-Language Navigation model
The challenging task of Vision-and-Language Navigation (VLN) requires embodied agents to follow natural language instructions to reach a goal location or object (e. g. `walk down the hallway and turn left at the piano').
Photorealistic Video Generation with Diffusion Models
We present W. A. L. T, a transformer-based approach for photorealistic video generation via diffusion modeling.
Ranked #1 on
Video Prediction
on Kinetics-600 12 frames, 64x64
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
