Search Results for author:
Found 47 papers, 25 papers with code
Unsupervised Volumetric Animation
We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects.
InfiniCity: Infinite-Scale City Synthesis
Toward infinite-scale 3D city synthesis, we propose a novel framework, InfiniCity, which constructs and renders an unconstrainedly large and 3D-grounded environment from random noises.
3DAvatarGAN: Bridging Domains for Personalized Editable Avatars
Finally, we propose a novel inversion method for 3D-GANs linking the latent spaces of the source and the target domains.
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
Existing 3D-aware image synthesis approaches mainly focus on generating a single canonical object and show limited capacity in composing a complex scene containing a variety of objects.
Real-Time Neural Light Field on Mobile Devices
Nevertheless, to reach a similar rendering quality as NeRF, the network in NeLF is designed with intensive computation, which is not mobile-friendly.
Rethinking Vision Transformers for MobileNet Size and Speed
With the success of Vision Transformers (ViTs) in computer vision tasks, recent arts try to optimize the performance and complexity of ViTs to enable efficient deployment on mobile devices.
LADIS: Language Disentanglement for 3D Shape Editing
Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision.
SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation
To enable interactive generation, our method supports a variety of input modalities that can be easily provided by a human, including images, text, partially observed shapes and combinations of these, further allowing to adjust the strength of each input.
Make-A-Story: Visual Memory Conditioned Consistent Story Generation
Our experiments for story generation on the MUGEN, the PororoSV and the FlintstonesSV dataset show that our method not only outperforms prior state-of-the-art in generating frames with high visual quality, which are consistent with the story, but also models appropriate correspondences between the characters and the background.
Affection: Learning Affective Explanations for Real-World Visual Data
To embark on this journey, we introduce and share with the research community a large-scale dataset that contains emotional reactions and free-form textual explanations for 85, 007 publicly available images, analyzed by 6, 283 annotators who were asked to indicate and explain how and why they felt in a particular way when observing a specific image, producing a total of 526, 749 responses.
Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training
Therefore, we analyze the feasibility and potentiality of using the layer freezing technique in sparse training and find it has the potential to save considerable training costs.
Cross-Modal 3D Shape Generation and Manipulation
In this paper, we propose a generic multi-modal generative model that couples the 2D modalities and implicit 3D representations through shared latent spaces.
EpiGRAF: Rethinking training of 3D GANs
In this work, we show that it is possible to obtain a high-resolution 3D generator with SotA image quality by following a completely different route of simply training the model patch-wise.
Discrete Contrastive Diffusion for Cross-Modal and Conditional Generation
To this end, we introduce a Conditional Discrete Contrastive Diffusion (CDCD) loss and design two contrastive diffusion mechanisms to effectively incorporate it into the denoising process.
EfficientFormer: Vision Transformers at MobileNet Speed
Our work proves that properly designed transformers can reach extremely low latency on mobile devices while maintaining high performance.
Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation
With the aim of obtaining interpretable and controllable scene representations, our model couples learnt scene-specific feature volumes with a scene agnostic neural rendering network.
Quantized GAN for Complex Music Generation from Dance Videos
We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates complex musical samples conditioned on dance videos.
R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis
On the other hand, Neural Light Field (NeLF) presents a more straightforward representation over NeRF in novel view synthesis -- the rendering of a pixel amounts to one single forward pass without ray-marching.
Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning
In addition, our model can extract visual information as suggested by the text prompt, e. g., "an object in image one is moving northeast", and generate corresponding videos.
Playable Environments: Video Manipulation in Space and Time
We present Playable Environments - a new representation for interactive video generation and manipulation in space and time.
F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization
Our approach achieves comparable and better performance, when compared not only to existing quantization techniques with INT32 multiplication or floating-point arithmetic, but also to the full-precision counterparts, achieving state-of-the-art performance.
NeROIC: Neural Rendering of Objects from Online Image Collections
We present a novel method to acquire object representations from online image collections, capturing high-quality geometry and material properties of arbitrary objects from photographs with varying cameras, illumination, and backgrounds.
InOut: Diverse Image Outpainting via GAN Inversion
Existing image outpainting methods pose the problem as a conditional image-to-image translation task, often generating repetitive structures and textures by replicating the content available in the input image.
StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2
We build our model on top of StyleGAN2 and it is just ${\approx}5\%$ more expensive to train at the same resolution while achieving almost the same image quality.
Flow Guided Transformable Bottleneck Networks for Motion Retargeting
Human motion retargeting aims to transfer the motion of one person in a "driving" video or set of images to another person.
A Good Image Generator Is What You Need for High-Resolution Video Synthesis
We introduce a motion generator that discovers the desired trajectory, in which content and motion are disentangled.
Motion Representations for Articulated Animation
To facilitate animation and prevent the leakage of the shape of the driving object, we disentangle shape and pose of objects in the region space.
Ranked #1 on
Video Reconstruction
on Tai-Chi-HD (512)
InfinityGAN: Towards Infinite-Pixel Image Synthesis
We present a novel framework, InfinityGAN, for arbitrary-sized image generation.
In&Out : Diverse Image Outpainting via GAN Inversion
Existing image outpainting methods pose the problem as a conditional image-to-image translation task, often generating repetitive structures and textures by replicating the content available in the input image.
SMIL: Multimodal Learning with Severely Missing Modality
A common assumption in multimodal learning is the completeness of training data, i. e., full modalities are available in all training examples.
Teachers Do More Than Teach: Compressing Image-to-Image Models
In this work, we aim to address these issues by introducing a teacher network that provides a search space in which efficient network architectures can be found, in addition to performing knowledge distillation.
Playable Video Generation
This paper introduces the unsupervised learning problem of playable video generation (PVG).
MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
In this paper, we present MichiGAN (Multi-Input-Conditioned Hair Image GAN), a novel conditional image generation method for interactive portrait hair manipulation.
Interactive Video Stylization Using Few-Shot Patch-Based Training
In this paper, we present a learning-based method to the keyframe-based video stylization that allows an artist to propagate the style from a few selected keyframes to the rest of the sequence.
Neural Hair Rendering
Unlike existing supervised translation methods that require model-level similarity to preserve consistent structure representation for both real images and fake renderings, our method adopts an unsupervised solution to work on arbitrary hair models.
Human Motion Transfer from Poses in the Wild
In this paper, we tackle the problem of human motion transfer, where we synthesize novel motion video for a target person that imitates the movement from a reference video.
Motion-supervised Co-Part Segmentation
To overcome this limitation, we propose a self-supervised deep learning method for co-part segmentation.
First Order Motion Model for Image Animation
To achieve this, we decouple appearance and motion information using a self-supervised formulation.
Ranked #1 on
Video Reconstruction
on Tai-Chi-HD
Task-Assisted Domain Adaptation with Anchor Tasks
Our key idea to improve domain adaptation is to introduce a separate anchor task (such as facial landmarks) whose annotations can be obtained at no cost or are already available on both synthetic and real datasets.
Transformable Bottleneck Networks
We propose a novel approach to performing fine-grained 3D manipulation of image content via a convolutional neural network, which we call the Transformable Bottleneck Network (TBN).
Train One Get One Free: Partially Supervised Neural Network for Bug Report Duplicate Detection and Clustering
Leveraging the assumption that learning the topic of a bug is a sub-task for detecting duplicates, we design a loss function that can jointly perform both tasks but needs supervision for only duplicate classification, achieving topic clustering in an unsupervised fashion.
3D Guided Fine-Grained Face Manipulation
This is achieved by first fitting a 3D face model and then disentangling the face into a texture and a shape.
Animating Arbitrary Objects via Deep Motion Transfer
This is achieved through a deep architecture that decouples appearance and motion information.
Hybrid VAE: Improving Deep Generative Models using Partial Observations
We show that such a combination is beneficial because the unlabeled data acts as a data-driven form of regularization, allowing generative models trained on few labeled samples to reach the performance of fully-supervised generative models trained on much larger datasets.
MoCoGAN: Decomposing Motion and Content for Video Generation
The proposed framework generates a video by mapping a sequence of random vectors to a sequence of video frames.
Self-Adaptive Matrix Completion for Heart Rate Estimation From Face Videos Under Realistic Conditions
Recent studies in computer vision have shown that, while practically invisible to a human observer, skin color changes due to blood flow can be captured on face videos and, surprisingly, be used to estimate the heart rate (HR).
Regressing a 3D Face Shape From a Single Image
To support the ability of our method to reliably reconstruct 3D shapes, we introduce a simple method for head pose estimation using a single image that reaches higher accuracy than the state of the art.
