Search Results for author:
Found 29 papers, 10 papers with code
Movie Gen: A Cast of Media Foundation Models
Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation.
Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation
Diffusion models are a powerful generative framework, but come with expensive inference.
Bespoke Non-Stationary Solvers for Fast Sampling of Diffusion and Flow Models
This paper introduces Bespoke Non-Stationary (BNS) Solvers, a solver distillation approach to improve sample efficiency of Diffusion and Flow models.
Animated Stickers: Bringing Stickers to Life with Video Diffusion
Our model is built on top of the state-of-the-art Emu text-to-image model, with the addition of temporal layers to model motion.
fMPI: Fast Novel View Synthesis in the Wild with Layered Scene Representations
In this study, we propose two novel input processing paradigms for novel view synthesis (NVS) methods based on layered scene representations that significantly improve their runtime without compromising quality.
Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models
Our findings provide insights into the efficiency of the conditional denoising process that contribute to more practical and swift deployment of text-conditioned diffusion models.
Bespoke Solvers for Generative Flow Models
For example, a Bespoke solver for a CIFAR10 model produces samples with Fr\'echet Inception Distance (FID) of 2. 73 with 10 NFE, and gets to 1% of the Ground Truth (GT) FID (2. 59) for this model with only 20 NFE.
BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis
To the best of our knowledge, this is the first approach that uses the reverse diffusion process to model full-body tracking as a conditional sequence generation task.
Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model
A particular challenge is that only a sparse tracking signal is available from standalone HMDs (Head Mounted Devices), often limited to tracking the user's head and wrists.
VisCo Grids: Surface Reconstruction with Viscosity and Coarea Grids
Surface reconstruction has been seeing a lot of progress lately by utilizing Implicit Neural Representations (INRs).
3D-CLFusion: Fast Text-to-3D Rendering with Contrastive Latent Diffusion
We note that the novelty of our model lies in that we introduce contrastive learning during training the diffusion prior which enables the generation of the valid view-invariant latent code.
Re-ReND: Real-time Rendering of NeRFs across Devices
Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines.
INGeo: Accelerating Instant Neural Scene Reconstruction with Noisy Geometry Priors
We present a method that accelerates reconstruction of 3D scenes and objects, aiming to enable instant reconstruction on edge devices such as mobile phones and AR/VR headsets.
3D-Aware Encoding for Style-based Neural Radiance Fields
In the first stage, we introduce a base encoder that converts the input image to a latent code.
Single-view 3D Body and Cloth Reconstruction under Complex Poses
We, therefore, propose a coarse-to-fine approach in which we first learn an implicit function that maps the input image to a 3D body shape with a low level of detail, but which correctly fits the underlying human pose, despite its complexity.
Ranked #1 on
3D Reconstruction
on 3DPeople
Enhancing Egocentric 3D Pose Estimation with Third Person Views
In this paper, we propose a novel approach to enhance the 3D body pose estimation of a person computed from videos captured from a single wearable camera.
PhysXNet: A Customizable Approach for LearningCloth Dynamics on Dressed People
We introduce PhysXNet, a learning-based approach to predict the dynamics of deformable clothes given 3D skeleton motion sequences of humans wearing these clothes.
FaceDet3D: Facial Expressions with 3D Geometric Detail Hallucination
Facial Expressions induce a variety of high-level details on the 3D face geometry.
H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction
In this paper, we tackle these limitations for the specific problem of few-shot full 3D head reconstruction, by endowing coordinate-based representations with a probabilistic shape prior that enables faster convergence and better generalization when using few input images (down to three).
SMPLicit: Topology-aware Generative Model for Clothed People
In this paper we introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry.
Ranked #2 on
Garment Reconstruction
on 4D-DRESS
(using extra training data)
FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction
The facial details are represented as a vertex displacement map and used then by a Neural Renderer to photo-realistically render novel images of any single image in any desired expression and view.
D-NeRF: Neural Radiance Fields for Dynamic Scenes
In this paper we introduce D-NeRF, a method that extends neural radiance fields to a dynamic domain, allowing to reconstruct and render novel images of objects under rigid and non-rigid motions from a \emph{single} camera moving around the scene.
C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds
Flow-based generative models have highly desirable properties like exact log-likelihood evaluation and exact latent-variable inference, however they are still in their infancy and have not received as much attention as alternative generative models.
3DPeople: Modeling the Geometry of Dressed Humans
Finally, we design a multi-resolution deep generative network that, given an input image of a dressed human, predicts his/her geometry image (and thus the clothed body shape) in an end-to-end manner.
Context-aware Human Motion Prediction
The problem of predicting human motion given a sequence of past observations is at the core of many applications in robotics and computer vision.
Fast video object segmentation with Spatio-Temporal GANs
To achieve this, we concentrate all the heavy computational load to the training phase with two critics that enforce spatial and temporal mask consistency over the last K frames.
Unsupervised Person Image Synthesis in Arbitrary Poses
Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen.
Geometry-Aware Network for Non-Rigid Shape Prediction from a Single View
We propose a method for predicting the 3D shape of a deformable surface from a single view.
GANimation: Anatomically-aware Facial Animation from a Single Image
Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis.
