Search Results for author:
Found 48 papers, 23 papers with code
PaLI-X: On Scaling up a Multilingual Vision and Language Model
We present the training recipe and results of scaling up PaLI-X, a multilingual vision and language model, both in terms of size of the components and the breadth of its training task mixture.
Ranked #1 on
Visual Question Answering (VQA)
on OK-VQA
Audiovisual Masked Autoencoders
Can we leverage the audiovisual information already present in video to improve self-supervised representation learning?
Ranked #1 on
Audio Classification
on EPIC-KITCHENS-100
(using extra training data)
RUST: Latent Neural Scene Representations from Unposed Imagery
Our main insight is that one can train a Pose Encoder that peeks at the target image and learns a latent pose embedding which is used by the decoder for view synthesis.
VCT: A Video Compression Transformer
The resulting video compression transformer outperforms previous methods on standard video compression data sets.
PolyViT: Co-training Vision Transformers on Images, Videos and Audio
Can we train a single transformer model capable of processing multiple modalities and datasets, whilst sharing almost all of its learnable parameters?
Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations
In this work, we propose the Scene Representation Transformer (SRT), a method which processes posed or unposed RGB images of a new area, infers a "set-latent scene representation", and synthesises novel views, all in a single feed-forward pass.
Revisiting the Calibration of Modern Neural Networks
Accurate estimation of predictive uncertainty (model calibration) is essential for the safe application of neural networks.
A Near-Optimal Algorithm for Debiasing Trained Machine Learning Models
We present a scalable post-processing algorithm for debiasing trained models, including deep neural networks (DNNs), which we prove to be near-optimal by bounding its excess Bayes risk.
MLP-Mixer: An all-MLP Architecture for Vision
Convolutional Neural Networks (CNNs) are the go-to model for computer vision.
Ranked #18 on
Image Classification
on OmniBenchmark
SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size
Before deploying machine learning models it is critical to assess their robustness.
Unconditional Synthesis of Complex Scenes Using a Semantic Bottleneck
Coupling the high-fidelity generation capabilities of label-conditional image synthesis methods with the flexibility of unconditional generative models, we propose a semantic bottleneck GAN model for unconditional synthesis of complex scenes.
A Near-Optimal Recipe for Debiasing Trained Machine Learning Models
We present an efficient and scalable algorithm for debiasing trained models, including deep neural networks (DNNs), which we prove to be near-optimal by bounding its excess Bayes risk.
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains.
A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation
The idea behind the \emph{unsupervised} learning of \emph{disentangled} representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms.
Which Model to Transfer? Finding the Needle in the Growing Haystack
Transfer learning has been recently popularized as a data-efficient alternative to training models from scratch, in particular for computer vision tasks where it provides a remarkably solid baseline.
Representation learning from videos in-the-wild: An object-centric approach
We propose a method to learn image representations from uncurated videos.
A Commentary on the Unsupervised Learning of Disentangled Representations
The goal of the unsupervised learning of disentangled representations is to separate the independent explanatory factors of variation in the data without access to supervision.
On Robustness and Transferability of Convolutional Neural Networks
Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts.
Self-Supervised Learning of Video-Induced Visual Invariances
We propose a general framework for self-supervised learning of transferable visual representations based on Video-Induced Visual Invariances (VIVI).
Ranked #15 on
Image Classification
on VTAB-1k
(using extra training data)
Semantic Bottleneck Scene Generation
For the former, we use an unconditional progressive segmentation generation network that captures the distribution of realistic semantic scene layouts.
Ranked #1 on
Image Generation
on Cityscapes-5K 256x512
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
And, how close are we to general visual representations?
Ranked #10 on
Image Classification
on VTAB-1k
(using extra training data)
The Visual Task Adaptation Benchmark
Representation learning promises to unlock deep learning for the long tail of vision tasks without expansive labelled datasets.
On Mutual Information Maximization for Representation Learning
Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different views of the data.
Precision-Recall Curves Using Information Divergence Frontiers
Despite the tremendous progress in the estimation of generative models, the development of tools for diagnosing their failures and assessing their performance has advanced at a much slower pace.
The GAN Landscape: Losses, Architectures, Regularization, and Normalization
Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion.
High-Fidelity Image Generation With Fewer Labels
Deep generative models are becoming a cornerstone of modern machine learning.
Ranked #10 on
Conditional Image Generation
on ImageNet 128x128
Conditional Image Generation
Vocal Bursts Intensity Prediction
Recent Advances in Autoencoder-Based Representation Learning
Finally, we provide an analysis of autoencoder-based representation learning through the lens of rate-distortion theory and identify a clear tradeoff between the amount of prior knowledge available about the downstream tasks, and how useful the representation is for this task.
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
The key idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms.
Self-Supervised GANs via Auxiliary Rotation Loss
In this work we exploit two popular unsupervised learning techniques, adversarial training and self-supervision, and take a step towards bridging the gap between conditional and unconditional GANs.
Ranked #6 on
Image Generation
on CelebA-HQ 128x128
On Self Modulation for Generative Adversarial Networks
Training Generative Adversarial Networks (GANs) is notoriously challenging.
A Large-Scale Study on Regularization and Normalization in GANs
Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion.
Assessing Generative Models via Precision and Recall
Recent advances in generative modeling have led to an increased interest in the study of statistical divergences as means of model comparison.
Deep Generative Models for Distribution-Preserving Lossy Compression
We propose and study the problem of distribution-preserving lossy compression.
Are GANs Created Equal? A Large-Scale Study
Generative adversarial networks (GAN) are a powerful subclass of generative models.
One-Shot Coresets: The Case of k-Clustering
Scaling clustering algorithms to massive data sets is a challenging task.
Stochastic Submodular Maximization: The Case of Coverage Functions
By exploiting that common extensions act linearly on the class of submodular functions, we employ projected stochastic gradient ascent and its variants in the continuous domain, and perform rounding to obtain discrete solutions.
Distributed and Provably Good Seedings for k-Means in Constant Rounds
The k-Means++ algorithm is the state of the art algorithm to solve k-Means clustering problems as the computed clusterings are O(log k) competitive in expectation.
Uniform Deviation Bounds for k-Means Clustering
In this paper, we provide a novel framework to obtain uniform deviation bounds for loss functions which are unbounded.
Training Gaussian Mixture Models at Scale via Coresets
In this work we show how to construct coresets for mixtures of Gaussians.
Practical Coreset Constructions for Machine Learning
We investigate coresets - succinct, small summaries of large data sets - so that solutions found on the summary are provably competitive with solution found on the full data set.
Uniform Deviation Bounds for Unbounded Loss Functions like k-Means
In this paper, we provide a novel framework to obtain uniform deviation bounds for loss functions which are *unbounded*.
Scalable k-Means Clustering via Lightweight Coresets
As such, they have been successfully used to scale up clustering models to massive data sets.
Fast and Provably Good Seedings for k-Means
Seeding - the task of finding initial cluster centers - is critical in obtaining high-quality clusterings for k-Means.
Horizontally Scalable Submodular Maximization
A variety of large-scale machine learning problems can be cast as instances of constrained submodular maximization.
Tradeoffs for Space, Time, Data and Risk in Unsupervised Learning
Using k-means clustering as a prototypical unsupervised learning problem, we show how we can strategically summarize the data (control space) in order to trade off risk and time when data is generated by a probabilistic model.
Strong Coresets for Hard and Soft Bregman Clustering with Applications to Exponential Family Mixtures
We propose a single, practical algorithm to construct strong coresets for a large class of hard and soft clustering problems based on Bregman divergences.
Fast and Robust Least Squares Estimation in Corrupted Linear Models
Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings.
