Search Results for author:
Found 29 papers, 11 papers with code
Cosmos World Foundation Model Platform for Physical AI
We position a world foundation model as a general-purpose world model that can be fine-tuned into customized world models for downstream applications.
Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Models
We introduce Edify Image, a family of diffusion models capable of generating photorealistic image content with pixel-perfect accuracy.
JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation
Personalized text-to-image generation models enable users to create images that depict their individual possessions in diverse scenes, finding applications in various domains.
DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
We fine-tune on a collection of fashion videos from the UBC Fashion dataset.
DreamPose: Fashion Video Synthesis with Stable Diffusion
We fine-tune on a collection of fashion videos from the UBC Fashion dataset.
SPACE: Speech-driven Portrait Animation with Controllable Expression
It uses a multi-stage approach, combining the controllability of facial landmarks with the high-quality synthesis power of a pretrained face generator.
Implicit Warping for Animation with Image Sets
We present a new implicit warping framework for image animation using sets of source images through the transfer of the motion of a driving video.
Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation
An effective approach is to supervise the training of deep neural networks with a high-fidelity dataset of desired input-output pairs, captured with a light stage.
Generating Long Videos of Dynamic Scenes
Existing video generation methods often fail to produce new content as a function of time while maintaining consistencies expected in real environments, such as plausible dynamics and object persistence.
BARCOR: Towards A Unified Framework for Conversational Recommendation Systems
CRS is a complex problem that consists of two main tasks: (1) recommendation and (2) response generation.
Multimodal Conditional Image Synthesis with Product-of-Experts GANs
Existing conditional image synthesis frameworks generate images based on user inputs in a single modality, such as text, segmentation, sketch, or style reference.
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
We propose a neural talking-head video synthesis model and demonstrate its application to video conferencing.
Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
The generative adversarial network (GAN) framework has emerged as a powerful tool for various image and video synthesis tasks, allowing the synthesis of visual content in an unconditional or input-conditional manner.
World-Consistent Video-to-Video Synthesis
This is because they lack knowledge of the 3D world being rendered and generate each frame only based on the past few frames.
Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter
Specifically, we directly treat the whole encoded feature map of the input texture as transposed convolution filters and the features' self-similarity map, which captures the auto-correlation information, as input to the transposed convolution.
Dancing to Music
In the analysis phase, we decompose a dance into a series of basic dance units, through which the model learns how to move.
Ranked #3 on
Motion Synthesis
on BRACE
Few-shot Video-to-Video Synthesis
To address the limitations, we propose a few-shot vid2vid framework, which learns to synthesize videos of previously unseen subjects or scenes by leveraging few example images of the target at test time.
Ranked #1 on
Video-to-Video Synthesis
on YouTube Dancing
Semantic Image Synthesis with Spatially-Adaptive Normalization
Previous methods directly feed the semantic layout as input to the deep network, which is then processed through stacks of convolution, normalization, and nonlinearity layers.
Ranked #3 on
Sketch-to-Image Translation
on COCO-Stuff
Partial Convolution based Padding
In this paper, we present a simple yet effective padding scheme that can be used as a drop-in module for existing convolutional neural networks.
Video-to-Video Synthesis
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.
Ranked #4 on
Video deraining
on Video Waterdrop Removal Dataset
Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation
Deep neural networks have largely failed to effectively utilize synthetic data when applied to real images due to the covariate shift problem.
Image Inpainting for Irregular Holes Using Partial Convolutions
Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value).
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs).
Ranked #2 on
Sketch-to-Image Translation
on COCO-Stuff
Conditional Image Generation
Fundus to Angiography Generation
+5
Light Field Video Capture Using a Learning-Based Hybrid Imaging System
Given a 3 fps light field sequence and a standard 30 fps 2D video, our system can then generate a full light field video at 30 fps.
Learning-Based View Synthesis for Light Field Cameras
Specifically, we propose a novel learning-based approach to synthesize new views from a sparse set of input views.
A 4D Light-Field Dataset and CNN Architectures for Material Recognition
We introduce a new light-field dataset of materials, and take advantage of the recent success of deep learning to perform material recognition on the 4D light-field.
SVBRDF-Invariant Shape and Reflectance Estimation From Light-Field Cameras
Light-field cameras have recently emerged as a powerful tool for one-shot passive 3D shape capture.
Depth From Semi-Calibrated Stereo and Defocus
In this work, we propose a multi-camera system where we combine a main high-quality camera with two low-res auxiliary cameras.
Occlusion-Aware Depth Estimation Using Light-Field Cameras
In this paper, we develop a depth estimation algorithm that treats occlusion explicitly; the method also enables identification of occlusion edges, which may be useful in other applications.
