Search Results for author:
Found 62 papers, 29 papers with code
Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models
With the emergence of text-to-video (T2V) diffusion models, its temporal counterpart, motion customization, has not yet been well investigated.
Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRM
We present Human-LRM, a diffusion-guided feed-forward model that predicts the implicit field of a human from a single image.
ActAnywhere: Subject-Aware Video Background Generation
Generating video background that tailors to foreground subject motion is an important problem for the movie industry and visual effects community.
ContactGen: Generative Contact Modeling for Grasp Generation
This paper presents a novel object-centric contact representation ContactGen for hand-object interaction.
Putting People in Their Place: Affordance-Aware Human Insertion into Scenes
Given a scene image with a marked region and an image of a person, we insert the person into the scene while respecting the scene affordances.
Normal-guided Garment UV Prediction for Human Re-texturing
To edit human videos in a physically plausible way, a texture map must take into account not only the garment transformation induced by the body movements and clothes fitting, but also its 3D fine-grained surface geometry.
Learning Visibility for Robust Dense Human Body Estimation
An alternative approach is to estimate dense vertices of a predefined template body in the image space.
Skeleton-free Pose Transfer for Stylized 3D Characters
We present the first method that automatically transfers poses between stylized 3D characters without skeletal rigging.
Audio-driven Neural Gesture Reenactment with Video Motion Graphs
We present a method that reenacts a high-quality video with gestures matching a target speech audio.
RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects
We test the effectiveness of our representation on the human image harmonization task by predicting shading that is coherent with a given background image.
The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation
Our framework leverages the best of non-parametric and model-based methods and is also robust to partial occlusion.
Ranked #1 on
3D Absolute Human Pose Estimation
on Human3.6M
(PA-MPJPE metric)
Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera
Appearance of dressed humans undergoes a complex geometric transformation induced not only by the static pose but also by its dynamics, i. e., there exists a number of cloth geometric configurations given a pose depending on the way it has moved.
Contact-Aware Retargeting of Skinned Motion
Self-contacts, such as when hands touch each other or the torso or the head, are important attributes of human body language and dynamics, yet existing methods do not model or preserve these contacts.
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
We present an algorithm for re-rendering a person from a single image under arbitrary poses.
Stochastic Scene-Aware Motion Prediction
A long-standing goal in computer vision is to capture, model, and realistically synthesize human behavior.
Single-image Full-body Human Relighting
We present a single-image data-driven method to automatically relight images with full-body humans in them.
Task-Generic Hierarchical Human Motion Prior using VAEs
We demonstrate the effectiveness of our hierarchical motion variational autoencoder in a variety of tasks including video-based human pose estimation, motion completion from partial observations, and motion synthesis from sparse key-frames.
Ranked #4 on
Motion Synthesis
on LaFAN1
HuMoR: 3D Human Motion Model for Robust Pose Estimation
We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape.
Attribute-conditioned Layout GAN for Automatic Graphic Design
In this paper, we introduce Attribute-conditioned Layout GAN to incorporate the attributes of design elements for graphic layout generation by forcing both the generator and the discriminator to meet attribute conditions.
Contact and Human Dynamics from Monocular Video
Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors that violate physical constraints, such as feet penetrating the ground and bodies leaning at extreme angles.
High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling
To address this challenge, we propose an iterative inpainting method with a feedback mechanism.
Ranked #6 on
Image Inpainting
on Places2
Generative Tweening: Long-term Inbetweening of 3D Human Motions
We introduce a biomechanically constrained generative adversarial network that performs long-term inbetweening of human motions, conditioned on keyframe constraints.
3D Ken Burns Effect from a Single Image
According to this depth estimate, our framework then maps the input image to a point cloud and synthesizes the resulting video frames by rendering the point cloud from the corresponding camera positions.
Ranked #4 on
Depth Estimation
on NYU-Depth V2
FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images
We show that methods trained on our dataset consistently perform well when tested on other datasets.
Ranked #8 on
3D Hand Pose Estimation
on FreiHAND
Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control
We experimentally demonstrate the strength of our approach over different non-hierarchical and hierarchical baselines.
LayoutGAN: Generating Graphic Layouts with Wireframe Discriminator
Layouts are important for graphic design and scene generation.
Multimodal Style Transfer via Graph Cuts
An assumption widely used in recent neural style transfer methods is that image styles can be described by global statics of deep features like Gram or covariance matrices.
LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators
Layout is important for graphic design and scene generation.
Foreground-aware Image Inpainting
We show that by such disentanglement, the contour completion model predicts reasonable contours of objects, and further substantially improves the performance of image inpainting.
On the Continuity of Rotation Representations in Neural Networks
Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn.
Learning to Sketch with Deep Q Networks and Demonstrated Strokes
Doodling is a useful and common intelligent skill that people can learn and master.
Flow-Grounded Spatial-Temporal Video Prediction from Still Images
Existing video prediction methods mainly rely on observing multiple historical frames or focus on predicting the next one-frame.
Free-Form Image Inpainting with Gated Convolution
We present a generative image inpainting system to complete images with free-form mask and guidance.
Ranked #3 on
Image Inpainting
on Places2 val
PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image
The proposed end-to-end DNN learns to directly infer a set of plane parameters and corresponding plane segmentation masks from a single RGB image.
Ranked #2 on
Plane Instance Segmentation
on NYU Depth v2
Neural Kinematic Networks for Unsupervised Motion Retargetting
We propose a recurrent neural network architecture with a Forward Kinematics layer and cycle consistency based adversarial training objective for unsupervised motion retargetting.
BodyNet: Volumetric Inference of 3D Human Body Shapes
Human shape estimation is an important task for video editing, animation and fashion industry.
Ranked #3 on
3D Human Pose Estimation
on Surreal
(using extra training data)
MAttNet: Modular Attention Network for Referring Expression Comprehension
In this paper, we address referring expression comprehension: localizing an image region described by a natural language expression.
Generalized Referring Expression Segmentation
Referring Expression
+1
Generative Image Inpainting with Contextual Attention
Motivated by these observations, we propose a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
Predicting Scene Parsing and Motion Dynamics in the Future
The ability of predicting the future is important for intelligent systems, e. g. autonomous vehicles and robots to plan early and make decisions accordingly.
FoveaNet: Perspective-aware Urban Scene Parsing
Thus, they suffer from heterogeneous object scales caused by perspective projection of cameras on actual scenes and inevitably encounter parsing failures on distant objects as well as other boundary and recognition errors.
3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks
The success of various applications including robotics, digital content creation, and visualization demand a structured and abstract representation of the 3D world from limited sensor data.
Material Editing Using a Physically Based Rendering Network
We propose an end-to-end network architecture that replicates the forward image formation process to accomplish this task.
Deep GrabCut for Object Selection
In this paper, we propose a novel segmentation approach that uses a rectangle as a soft constraint by transforming it into an Euclidean distance map.
Decomposing Motion and Content for Natural Video Sequence Prediction
To the best of our knowledge, this is the first end-to-end trainable network architecture with motion and content separation to model the spatiotemporal dynamics for pixel-level future prediction in natural videos.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Universal Style Transfer via Feature Transforms
The whitening and coloring transforms reflect a direct matching of feature covariance of the content image to a given style image, which shares similar spirits with the optimization of Gram matrix based cost in neural style transfer.
Generative Face Completion
In this paper, we propose an effective face completion algorithm using a deep generative model.
Learning to Generate Long-term Future via Hierarchical Prediction
To avoid inherent compounding errors in recursive pixel-level prediction, we propose to first estimate high-level structure in the input frames, then predict how that structure evolves in the future, and finally by observing a single frame from the past and the predicted high-level structure, we construct the future frames without having to observe any of the pixel-level predictions.
Forecasting Human Dynamics from Static Images
This paper presents the first study on forecasting human dynamics from static images.
Recurrent Multimodal Interaction for Referring Image Segmentation
In this paper we are interested in the problem of image segmentation given natural language descriptions, i. e. referring expressions.
Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
Instead of taking a 'blank slate' approach, we first explicitly infer the parts of the geometry visible both in the input and novel views and then re-cast the remaining synthesis problem as image completion.
Diversified Texture Synthesis with Feed-forward Networks
Recent progresses on deep discriminative and generative modeling have shown promising results on texture synthesis.
Video Scene Parsing with Predictive Feature Learning
In this way, the network can effectively learn to capture video dynamics and temporal context, which are critical clues for video scene parsing, without requiring extra manual annotations.
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
We demonstrate the ability of the model in generating 3D volume from a single 2D image with three sets of experiments: (1) learning from single-class objects; (2) learning from multi-class objects and (3) testing on novel object classes.
Object Tracking via Dual Linear Structured SVM and Explicit Feature Map
Structured support vector machine (SSVM) based methods has demonstrated encouraging performance in recent object tracking benchmarks.
Object Contour Detection with a Fully Convolutional Encoder-Decoder Network
We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network.
Deep Interactive Object Selection
Interactive object selection is a very important research problem and has many applications.
Ranked #11 on
Interactive Segmentation
on SBD
Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
An important problem for both graphics and vision is to synthesize novel views of a 3D object from a single image.
Attribute2Image: Conditional Image Generation from Visual Attributes
This paper investigates a novel problem of generating images from visual attributes.
Multi-Objective Convolutional Learning for Face Labeling
This paper formulates face labeling as a conditional random field with unary and pairwise classifiers.
PatchCut: Data-Driven Object Segmentation via Local Shape Transfer
The transferred local shape masks constitute a patch-level segmentation solution space and we thus develop a novel cascade algorithm, PatchCut, for coarse-to-fine object segmentation.
Context Driven Scene Parsing with Attention to Rare Classes
This paper presents a scalable scene parsing algorithm based on image retrieval and superpixel matching.
Max-Margin Boltzmann Machines for Object Segmentation
We present Max-Margin Boltzmann Machines (MMBMs) for object segmentation.
