Search Results for author:
Found 21 papers, 10 papers with code
Video-to-Video Synthesis
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.
Few-shot Video-to-Video Synthesis
To address the limitations, we propose a few-shot vid2vid framework, which learns to synthesize videos of previously unseen subjects or scenes by leveraging few example images of the target at test time.
Ranked #1 on
Video-to-Video Synthesis
on YouTube Dancing
SDCNet: Video Prediction Using Spatially-Displaced Convolution
We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.
SDC-Net: Video prediction using spatially-displaced convolution
We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.
Ranked #1 on
Video Prediction
on YouTube-8M
Image Inpainting for Irregular Holes Using Partial Convolutions
Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value).
Partial Convolution based Padding
In this paper, we present a simple yet effective padding scheme that can be used as a drop-in module for existing convolutional neural networks.
DiffiT: Diffusion Vision Transformers for Image Generation
In this paper, we study the effectiveness of ViTs in diffusion-based generative learning and propose a new model denoted as Diffusion Vision Transformers (DiffiT).
Ranked #3 on
Image Generation
on ImageNet 256x256
Unsupervised Video Interpolation Using Cycle Consistency
We further introduce a pseudo supervised loss term that enforces the interpolated frames to be consistent with predictions of a pre-trained interpolation model.
Ranked #1 on
Video Frame Interpolation
on UCF101
(PSNR (sRGB) metric)
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision
We introduce DiscoBox, a novel framework that jointly learns instance segmentation and semantic correspondence using bounding box supervision.
Dual Contrastive Loss and Attention for GANs
Lastly, we study different attention architectures in the discriminator, and propose a reference attention mechanism.
Material Editing Using a Physically Based Rendering Network
We propose an end-to-end network architecture that replicates the forward image formation process to accomplish this task.
Symmetry-aware Depth Estimation using Deep Neural Networks
Due to the abundance of 2D product images from the Internet, developing efficient and scalable algorithms to recover the missing depth information is central to many applications.
Neural Inverse Rendering of an Indoor Scene from a Single Image
Inverse rendering aims to estimate physical attributes of a scene, e. g., reflectance, geometry, and lighting, from image(s).
Dual-Space Decomposition of 2D Complex Shapes
In this paper, we propose a new decomposition method, called Dual-space Decomposition that handles complex 2D shapes by recognizing the importance of holes and classifying holes as either topological noise or structurally important features.
Continuous Visibility Feature
We say that a point q on the mesh is continuously visible from another point p if there exists a geodesic path connecting p and q that is entirely visible by p. In order to efficiently estimate the continuous visibility for all the vertices in a model, we propose two approaches that use specific CVF properties to avoid exhaustive visibility tests.
Panoptic-based Image Synthesis
Conditional image synthesis for generating photorealistic images serves various applications for content editing to content generation.
Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter
Specifically, we directly treat the whole encoded feature map of the input texture as transposed convolution filters and the features' self-similarity map, which captures the auto-correlation information, as input to the transposed convolution.
Neural FFTs for Universal Texture Image Synthesis
The conventional CNNs, recently adopted for synthesis, require to train and test on the same set of images and fail to generalize to unseen images.
View Generalization for Single Image Textured 3D Models
We describe a cycle consistency loss that encourages model textures to be aligned, so as to encourage sharing.
Coupled Segmentation and Edge Learning via Dynamic Graph Propagation
It is therefore interesting to study how these two tasks can be coupled to benefit each other.
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
Despite tremendous progress in generating high-quality images using diffusion models, synthesizing a sequence of animated frames that are both photorealistic and temporally coherent is still in its infancy.
Ranked #8 on
Text-to-Video Generation
on UCF-101
