Search Results for author:
Found 21 papers, 7 papers with code
Perception Test: A Diagnostic Benchmark for Multimodal Video Models
We propose a novel multimodal video benchmark - the Perception Test - to evaluate the perception and reasoning skills of pre-trained multimodal models (e. g. Flamingo, BEiT-3, or GPT-4).
TAP-Vid: A Benchmark for Tracking Any Point in a Video
Generic motion understanding from video involves not only tracking objects, but also perceiving how their surfaces deform and move.
Self-supervised video pretraining yields strong image representations
Videos contain far more information than still images and hold the potential for learning rich representations of the visual world.
Where Should I Spend My FLOPS? Efficiency Evaluations of Visual Pre-training Methods
Self-supervised methods have achieved remarkable success in transfer learning, often achieving the same or better accuracy than supervised pre-training.
Transframer: Arbitrary Frame Prediction with Generative Models
We present a general-purpose framework for image modelling and vision tasks based on probabilistic frame prediction.
Object discovery and representation networks
The promise of self-supervised learning (SSL) is to leverage large amounts of unlabeled data to solve complex tasks.
General-purpose, long-context autoregressive modeling with Perceiver AR
Real-world data is high-dimensional: a book, image, or musical performance can easily contain hundreds of thousands of elements even after compression.
Input-level Inductive Biases for 3D Reconstruction
Much of the recent progress in 3D vision has been driven by the development of specialized architectures that incorporate geometrical inductive biases.
Efficient Visual Pretraining with Contrastive Detection
Self-supervised pretraining has been shown to yield powerful representations for transfer learning.
Ranked #48 on
Semantic Segmentation
on Cityscapes val
(using extra training data)
A Short Note on the Kinetics-700-2020 Human Action Dataset
We describe the 2020 edition of the DeepMind Kinetics human action dataset, which replenishes and extends the Kinetics-700 dataset.
The AVA-Kinetics Localized Human Actions Video Dataset
The dataset is collected by annotating videos from the Kinetics-700 dataset using the AVA annotation protocol, and extending the original AVA dataset with these new AVA annotated Kinetics clips.
Visual Grounding in Video for Unsupervised Word Translation
Given this shared embedding we demonstrate that (i) we can map words between the languages, particularly the 'visual' words; (ii) that the shared embedding provides a good initialization for existing unsupervised text-based word translation techniques, forming the basis for our proposed hybrid visual-text mapping algorithm, MUVE; and (iii) our approach achieves superior performance by addressing the shortcomings of text-based methods -- it is more robust, handles datasets with less commonality, and is applicable to low-resource languages.
Controllable Attention for Structured Layered Video Decomposition
The objective of this paper is to be able to separate a video into its natural layers, and to control which of the separated layers to attend to.
Video Action Transformer Network
We introduce the Action Transformer model for recognizing and localizing human actions in video clips.
Ranked #5 on
Action Recognition
on AVA v2.1
The Visual Centrifuge: Model-Free Layered Video Representations
True video understanding requires making sense of non-lambertian scenes where the color of light arriving at the camera sensor encodes information about not just the last object it collided with, but about multiple mediums -- colored windows, dirty mirrors, smoke or rain.
A Better Baseline for AVA
We introduce a simple baseline for action localization on the AVA dataset.
Ranked #11 on
Action Recognition
on AVA v2.1
Shape and Symmetry Induction for 3D Objects
Actions as simple as grasping an object or navigating around it require a rich understanding of that object's 3D shape from a given viewpoint.
Amodal Completion and Size Constancy in Natural Scenes
We consider the problem of enriching current object detection systems with veridical object sizes and relative depth estimates from a single image.
Pose Induction for Novel Object Categories
We address the task of predicting pose for objects of unannotated object categories from a small seed set of annotated object classes.
Virtual View Networks for Object Reconstruction
All that structure from motion algorithms "see" are sets of 2D points.
Category-Specific Object Reconstruction from a Single Image
Object reconstruction from a single image -- in the wild -- is a problem where we can make progress and get meaningful results today.
