Search Results for author:
Found 30 papers, 20 papers with code
Improve Supervised Representation Learning with Masked Image Modeling
On K-Nearest-Neighbor image retrieval evaluation with ImageNet-1k, the same model outperforms the baseline by 1. 32%.
Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency
We demonstrate image generation and captioning performance on par with state-of-the-art text-to-image and image-to-text models with orders of magnitude fewer (only 3M) paired image-text data.
StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners
We investigate the potential of learning visual representations using synthetic images generated by text-to-image models.
Learning Disentangled Prompts for Compositional Image Synthesis
We study domain-adaptive image synthesis, the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images, to better understand the compositional image synthesis.
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts.
Score-Based Diffusion Models as Principled Priors for Inverse Imaging
In this work, we empirically validate the theoretically-proven probability function of a score-based diffusion model.
Soundini: Sound-Guided Diffusion for Natural Video Editing
We propose a method for adding sound-guided visual effects to specific regions of videos with a zero-shot setting.
StraIT: Non-autoregressive Generation with Stratified Image Transformer
We propose Stratified Image Transformer(StraIT), a pure non-autoregressive(NAR) generative model that demonstrates superiority in high-quality image synthesis over existing autoregressive(AR) and diffusion models(DMs).
VQ3D: Learning a 3D-Aware Generative Model on ImageNet
Recent work has shown the possibility of training generative models of 3D content from 2D image collections on small datasets corresponding to a single object class, such as human faces, animal faces, or cars.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
MAGVIT: Masked Generative Video Transformer
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model.
Ranked #1 on
Video Prediction
on Something-Something V2
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis
In this work, we propose MAsked Generative Encoder (MAGE), the first framework to unify SOTA image generation and self-supervised representation learning.
Ranked #2 on
Unconditional Image Generation
on ImageNet 256x256
A simple, efficient and scalable contrastive masked autoencoder for learning visual representations
Our framework is a minimal and conceptually clean synthesis of (C) contrastive learning, (A) masked autoencoders, and (N) the noise prediction approach used in diffusion models.
Imagic: Text-Based Real Image Editing with Diffusion Models
In this paper we demonstrate, for the very first time, the ability to apply complex (e. g., non-rigid) text-guided semantic edits to a single real image.
Visual Prompt Tuning for Generative Transfer Learning
We base our framework on state-of-the-art generative vision transformers that represent an image as a sequence of visual tokens to the autoregressive or non-autoregressive transformers.
Improved Masked Image Generation with Token-Critic
Given a masked-and-reconstructed real image, the Token-Critic model is trained to distinguish which visual tokens belong to the original image and which were sampled by the generative transformer.
MaskGIT: Masked Generative Image Transformer
At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation.
Ranked #2 on
Text-to-Image Generation
on LHQC
BLT: Bidirectional Layout Transformer for Controllable Layout Generation
During inference, BLT first generates a draft layout from the input and then iteratively refines it into a high-quality layout by masking out low-confident attributes.
Pyramid Adversarial Training Improves ViT Performance
In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall performance.
Ranked #9 on
Domain Generalization
on ImageNet-C
(using extra training data)
Palette: Image-to-Image Diffusion Models
We expect this standardized evaluation protocol to play a role in advancing image-to-image translation research.
Ranked #1 on
Colorization
on ImageNet ctest10k
SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting
We present SLIDE, a modular and unified system for single image 3D photography that uses a simple yet effective soft layering strategy to better preserve appearance details in novel views.
ViTGAN: Training GANs with Vision Transformers
Recently, Vision Transformers (ViTs) have shown competitive performance on image recognition while requiring less vision-specific inductive biases.
Ranked #76 on
Image Generation
on CIFAR-10
OCONet: Image Extrapolation by Object Completion
Existing methods struggle to extrapolate images with salient objects in the foreground or are limited to very specific objects such as humans, but tend to work well on indoor/outdoor scenes.
LASR: Learning Articulated Shape Reconstruction from a Monocular Video
Remarkable progress has been made in 3D reconstruction of rigid structures from a video or a collection of images.
AutoFlow: Learning a Better Training Set for Optical Flow
Synthetic datasets play a critical role in pre-training CNN models for optical flow, but they are painstaking to generate and hard to adapt to new applications.
Deep 3D-to-2D Watermarking: Embedding Messages in 3D Meshes and Extracting Them from 2D Renderings
Digital watermarking is widely used for copyright protection.
Zoom-to-Inpaint: Image Inpainting with High-Frequency Details
Although deep learning has enabled a huge leap forward in image inpainting, current methods are often unable to synthesize realistic high-frequency details.
Distortion Agnostic Deep Watermarking
Watermarking is the process of embedding information into an image that can survive under distortions, while requiring the encoded image to have little or no perceptual difference from the original image.
SwapNet: Garment Transfer in Single View Images
Garment transfer is a challenging task that requires (i) disentangling the features of the clothing from the body pose and shape and (ii) realistic synthesis of the garment texture on the new body.
Ranked #1 on
Virtual Try-on
on FashionIQ
(using extra training data)
PairedCycleGAN: Asymmetric Style Transfer for Applying and Removing Makeup
This paper introduces an automatic method for editing a portrait photo so that the subject appears to be wearing makeup in the style of another person in a reference photo.
