Search Results for author:
Found 40 papers, 24 papers with code
Learning Vision from Models Rivals Learning Vision from Data
We introduce SynCLR, a novel approach for learning visual representations exclusively from synthetic images and synthetic captions, without any real data.
Scaling Laws of Synthetic Images for Model Training ... for Now
Our findings also suggest that scaling synthetic data can be particularly effective in scenarios such as: (1) when there is a limited supply of real images for a supervised problem (e. g., fewer than 0. 5 million images in ImageNet), (2) when the evaluation dataset diverges significantly from the training data, indicating the out-of-distribution scenario, or (3) when synthetic data is used in conjunction with real images, as demonstrated in the training of CLIP models.
Improve Supervised Representation Learning with Masked Image Modeling
On K-Nearest-Neighbor image retrieval evaluation with ImageNet-1k, the same model outperforms the baseline by 1. 32%.
Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency
We demonstrate image generation and captioning performance on par with state-of-the-art text-to-image and image-to-text models with orders of magnitude fewer (only 3M) paired image-text data.
Substance or Style: What Does Your Image Embedding Know?
We find that image-text models (CLIP and ALIGN) are better at recognizing new examples of style transfer than masking-based models (CAN and MAE).
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts.
StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners
We investigate the potential of learning visual representations using synthetic images generated by text-to-image models.
Improving CLIP Training with Language Rewrites
During training, LaCLIP randomly selects either the original texts or the rewritten versions as text augmentations for each image.
Steerable Equivariant Representation Learning
In this paper, we propose a method of learning representations that are instead equivariant to data augmentations.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis
In this work, we propose MAsked Generative Encoder (MAGE), the first framework to unify SOTA image generation and self-supervised representation learning.
Ranked #2 on
Unconditional Image Generation
on ImageNet 256x256
A simple, efficient and scalable contrastive masked autoencoder for learning visual representations
Our framework is a minimal and conceptually clean synthesis of (C) contrastive learning, (A) masked autoencoders, and (N) the noise prediction approach used in diffusion models.
Simplified Transfer Learning for Chest Radiography Models Using Less Data
Supervised contrastive learning enabled performance comparable to state-of-the-art deep learning models in multiple clinical tasks by using as few as 45 images and is a promising method for predictive modeling with use of small data sets and for predicting outcomes in shifting patient populations.
Object-Aware Cropping for Self-Supervised Learning
This assumption is mostly satisfied in datasets such as ImageNet where there is a large, centered object, which is highly likely to be present in random crops of the full image.
Pyramid Adversarial Training Improves ViT Performance
In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall performance.
Ranked #9 on
Domain Generalization
on ImageNet-C
(using extra training data)
CSI: Contrastive Data Stratification for Interaction Prediction and its Application to Compound-Protein Interaction Prediction
Accurately predicting the likelihood of interaction between two objects (compound-protein sequence, user-item, author-paper, etc.)
Unsupervised Disentanglement without Autoencoding: Pitfalls and Future Directions
Disentangled visual representations have largely been studied with generative models such as Variational AutoEncoders (VAEs).
Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers
We use our reconstruction model as a tool for exploring the nature of representations, including: the influence of model architecture and training objectives (specifically robust losses), the forms of invariance that networks achieve, representational differences between correctly and incorrectly classified images, and the effects of manipulating logits and images.
What Makes for Good Views for Contrastive Learning?
Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning.
Ranked #2 on
Contrastive Learning
on imagenet-1k
Supervised Contrastive Learning
Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models.
Ranked #2 on
Class Incremental Learning
on cifar100
Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?
The focus of recent meta-learning research has been on the development of learning algorithms that can quickly adapt to test time tasks with limited data and low computational cost.
Fantastic Generalization Measures and Where to Find Them
We present the first large scale study of generalization in deep networks.
Contrastive Representation Distillation
We demonstrate that this objective ignores important structural knowledge of the teacher network.
Ranked #13 on
Knowledge Distillation
on CIFAR-100
Boundless: Generative Adversarial Networks for Image Extension
Image extension models have broad applications in image editing, computational photography and computer graphics.
Ranked #2 on
Uncropping
on Places2 val
Adversarial Robustness through Local Linearization
Using this regularizer, we exceed current state of the art and achieve 47% adversarial accuracy for ImageNet with l-infinity adversarial perturbations of radius 4/255 under an untargeted, strong, white-box attack.
Contrastive Multiview Coding
We analyze key properties of the approach that make it work, finding that the contrastive loss outperforms a popular alternative based on cross-view prediction, and that the more views we learn from, the better the resulting representation captures underlying scene semantics.
Ranked #48 on
Self-Supervised Action Recognition
on UCF101
A Closed-Form Learned Pooling for Deep Classification Networks
This operator can learn a strict super-set of what can be learned by average pooling or convolutions.
Predicting the Generalization Gap in Deep Networks with Margin Distributions
In this paper, we propose such a measure, and conduct extensive empirical studies on how well it can predict the generalization gap.
Sparse, Smart Contours to Represent and Edit Images
We study the problem of reconstructing an image from information stored at contour locations.
Large Margin Deep Networks for Classification
We present a formulation of deep learning that aims at producing a large margin classifier.
Smart, Sparse Contours to Represent and Edit Images
We study the problem of reconstructing an image from information stored at contour locations.
Synthesizing Normalized Faces from Facial Identity Features
We present a method for synthesizing a frontal, neutral-expression image of a person's face given an input face photograph.
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Collecting well-annotated image datasets to train modern machine learning algorithms is prohibitively expensive for many tasks.
Generative Adversarial Network
Unsupervised Domain Adaptation
Domain Separation Networks
However, by focusing only on creating a mapping or shared representation between the two domains, they ignore the individual characteristics of each domain.
Ranked #1 on
Domain Adaptation
on Synth Objects-to-LINEMOD
Learning Ordinal Relationships for Mid-Level Vision
We demonstrate that this frame- work works well on two important mid-level vision tasks: intrinsic image decomposition and depth from an RGB im- age.
Learning visual groups from co-occurrences in space and time
We propose a self-supervised framework that learns to group visual entities based on their rate of co-occurrence in space and time.
Reflection Removal Using Ghosting Cues
For single-pane windows, ghosting cues arise from shifted reflections on the two surfaces of the glass pane.
Shape and Illumination from Shading using the Generic Viewpoint Assumption
The Generic Viewpoint Assumption (GVA) states that the position of the viewer or the light in a scene is not special.
Blind Deconvolution with Non-local Sparsity Reweighting
Blind deconvolution has made significant progress in the past decade.
Fast Image Deconvolution using Hyper-Laplacian Priors
In this paper we describe a deconvolution approach that is several orders of magnitude faster than existing techniques that use hyper-Laplacian priors.
