Search Results for author:
Found 64 papers, 54 papers with code
Aligning and Projecting Images to Class-conditional Generative Networks
We present a method for projecting an input image into the space of a class-conditional generative neural network.
Controllable Visual-Tactile Synthesis
Deep generative models have various content creation applications such as graphic design, e-commerce, and virtual Try-on.
Generalizing Dataset Distillation via Deep Generative Prior
Dataset Distillation aims to distill an entire dataset's knowledge into a few synthetic images.
Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis
Given a minute-long RGBD video of people interacting with their pets, we render the scene from novel camera trajectories derived from in-scene motion of actors: (1) egocentric cameras that simulate the point of view of a target actor and (2) 3rd-person cameras that follow the actor.
Expressive Text-to-Image Generation with Rich Text
For each region, we enforce its text attributes by creating region-specific detailed prompts and applying region-specific guidance.
Ablating Concepts in Text-to-Image Diffusion Models
To achieve this goal, we propose an efficient method of ablating concepts in the pretrained model, i. e., preventing the generation of a target concept.
Scaling up GANs for Text-to-Image Synthesis
From a technical standpoint, it also marked a drastic change in the favored architecture to design generative image models.
Ranked #8 on
Image Generation
on ImageNet 256x256
3D-aware Conditional Image Synthesis
We propose pix2pix3D, a 3D-aware conditional generative model for controllable photorealistic image synthesis.
Zero-shot Image-to-Image Translation
However, it is still challenging to directly apply these models for editing real images for two reasons.
Domain Expansion of Image Generators
First, we note the generator contains a meaningful, pretrained latent space.
Multi-Concept Customization of Text-to-Image Diffusion
Can we teach a model to quickly acquire a new concept, given a few examples?
Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models
In this work, we present Spatially Sparse Inference (SSI), a general-purpose technique that selectively performs computation for edited regions and accelerates various generative models, including both conditional GANs and diffusion models.
Content-Based Search for Deep Generative Models
The growing proliferation of pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence.
Ranked #1 on
Model Description Based Search
on Generative Models
Image and Sketch based Model Retrieval
Model Description Based Search
+2
Rewriting Geometric Rules of a GAN
Our method allows a user to create a model that synthesizes endless objects with defined geometric changes, enabling the creation of a new generative model without the burden of curating a large-scale dataset.
Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing
We propose a new method to invert and edit such complex images in the latent space of GANs, such as StyleGAN2.
Dataset Distillation by Matching Training Trajectories
To efficiently obtain the initial and target network parameters for large-scale datasets, we pre-compute and store training trajectories of expert networks trained on the real dataset.
Ensembling Off-the-shelf Models for GAN Training
Can the collective "knowledge" from a large bank of pretrained vision models be leveraged to improve GAN training?
Ranked #1 on
Image Generation
on AFHQ Cat
GAN-Supervised Dense Visual Alignment
We propose GAN-Supervised Learning, a framework for learning discriminative models and their GAN-generated training data jointly end-to-end.
Contrastive Feature Loss for Image Prediction
Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result.
Sketch Your Own GAN
In particular, we change the weights of an original GAN model according to user sketches.
SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
The key challenge is balancing faithfulness to the user input (e. g., hand-drawn colored strokes) and realism of the synthesized image.
Depth-supervised NeRF: Fewer Views and Faster Training for Free
Crucially, SFM also produces sparse 3D points that can be used as "free" depth supervision during training: we add a loss to encourage the distribution of a ray's terminating depth matches a given 3D keypoint, incorporating depth uncertainty.
Editing Conditional Radiance Fields
In this paper, we explore enabling user editing of a category-level NeRF - also known as a conditional radiance field - trained on a shape category.
Ranked #1 on
Novel View Synthesis
on PhotoShape
Ensembling with Deep Generative Views
Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification.
On Aliased Resizing and Surprising Subtleties in GAN Evaluation
Furthermore, we show that if compression is used on real training images, FID can actually improve if the generated images are also subsequently compressed.
Anycost GANs for Interactive Image Synthesis and Editing
Generative adversarial networks (GANs) have enabled photorealistic image synthesis and editing.
Understanding the Role of Individual Units in a Deep Neural Network
Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes.
The Hessian Penalty: A Weak Prior for Unsupervised Disentanglement
In this paper, we propose the Hessian Penalty, a simple regularization term that encourages the Hessian of a generative model with respect to its input to be diagonal.
Rewriting a Deep Generative Model
To address the problem, we propose a formulation in which the desired rule is changed by manipulating a layer of a deep network as a linear associative memory.
Contrastive Learning for Unpaired Image-to-Image Translation
Furthermore, we draw negatives from within the input image itself, rather than from the rest of the dataset.
Swapping Autoencoder for Deep Image Manipulation
Deep generative models have become increasingly effective at producing realistic images from randomly sampled seeds, but using such models for controllable manipulation of existing images remains challenging.
Differentiable Augmentation for Data-Efficient GAN Training
Furthermore, with only 20% training data, we can match the top performance on CIFAR-10 and CIFAR-100.
Ranked #1 on
Image Generation
on CIFAR-10 (20% data)
Diverse Image Generation via Self-Conditioned GANs
We introduce a simple but effective unsupervised method for generating realistic and diverse images.
Semantic Photo Manipulation with a Generative Image Prior
First, it is hard for GANs to precisely reproduce an input image.
Transforming and Projecting Images into Class-conditional Generative Networks
We present a method for projecting an input image into the space of a class-conditional generative neural network.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
GAN Compression: Efficient Architectures for Interactive Conditional GANs
Directly applying existing compression methods yields poor performance due to the difficulty of GAN training and the differences in generator architectures.
Seeing What a GAN Cannot Generate
Differences in statistics reveal object classes that are omitted by a GAN.
Connecting Touch and Vision via Cross-Modal Prediction
To connect vision and touch, we introduce new tasks of synthesizing plausible tactile signals from visual inputs as well as imagining how we interact with objects given tactile data as input.
Learning the signatures of the human grasp using a scalable tactile glove
Using a low-cost (about US$10) scalable tactile glove sensor array, we record a large-scale tactile dataset with 135, 000 frames, each covering the full hand, while interacting with 26 different objects.
Visualizing and Understanding GANs
We present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level.
Semantic Image Synthesis with Spatially-Adaptive Normalization
Previous methods directly feed the semantic layout as input to the deep network, which is then processed through stacks of convolution, normalization, and nonlinearity layers.
On the Units of GANs (Extended Abstract)
We quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output.
Visual Object Networks: Image Generation with Disentangled 3D Representation
Our model first learns to synthesize 3D shapes that are indistinguishable from real shapes.
Visual Object Networks: Image Generation with Disentangled 3D Representations
The VON not only generates images that are more realistic than the state-of-the-art 2D image synthesis methods but also enables many 3D operations such as changing the viewpoint of a generated image, shape and texture editing, linear interpolation in texture and shape space, and transferring appearance across different objects and viewpoints.
Dataset Distillation
Model distillation aims to distill the knowledge of a complex model into a simpler one.
GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
Then, we quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output.
Propagation Networks for Model-Based Control Under Partial Observation
There has been an increasing interest in learning dynamics simulators for model-based control.
3D-Aware Scene Manipulation via Inverse Graphics
In this work, we propose 3D scene de-rendering networks (3D-SDN) to address the above issues by integrating disentangled representations for semantics, geometry, and appearance into a deep generative model.
Video-to-Video Synthesis
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.
Generating Adversarial Examples with Adversarial Networks
A challenge to explore adversarial robustness of neural networks on MNIST.
Spatially Transformed Adversarial Examples
Perturbations generated through spatial transformation could result in large $\mathcal{L}_p$ distance measures, but our extensive experiments show that such spatially transformed adversarial examples are perceptually realistic and more difficult to defend against with existing defense systems.
Toward Multimodal Image-to-Image Translation
Our proposed method encourages bijective consistency between the latent encoding and output modes.
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs).
Ranked #2 on
Sketch-to-Image Translation
on COCO-Stuff
Conditional Image Generation
Fundus to Angiography Generation
+4
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Domain adaptation is critical for success in new, unseen environments.
Real-Time User-Guided Image Colorization with Learned Deep Priors
The system directly maps a grayscale image, along with sparse, local user "hints" to an output colorization with a Convolutional Neural Network (CNN).
Light Field Video Capture Using a Learning-Based Hybrid Imaging System
Given a 3 fps light field sequence and a standard 30 fps 2D video, our system can then generate a full light field video at 30 fps.
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs.
Ranked #1 on
Image-to-Image Translation
on zebra2horse
(Frechet Inception Distance metric)
Multimodal Unsupervised Image-To-Image Translation
Style Transfer
+2
Image-to-Image Translation with Conditional Adversarial Networks
We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems.
Generative Visual Manipulation on the Natural Image Manifold
Realistic image manipulation is challenging because it requires modifying the image appearance in a user-controlled way, while preserving the realism of the result.
A 4D Light-Field Dataset and CNN Architectures for Material Recognition
We introduce a new light-field dataset of materials, and take advantage of the recent success of deep learning to perform material recognition on the 4D light-field.
Learning a Discriminative Model for the Perception of Realism in Composite Images
What makes an image appear realistic?
MILCut: A Sweeping Line Multiple Instance Learning Paradigm for Interactive Image Segmentation
Interactive segmentation, in which a user provides a bounding box to an object of interest for image segmentation, has been applied to a variety of applications in image editing, crowdsourcing, computer vision, and medical imaging.
