Search Results for author:
Found 57 papers, 29 papers with code
Example-based Motion Synthesis via Generative Motion Matching
At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches.
Towards Robust Probabilistic Modeling on SO(3) via Rotation Laplace Distribution
With this benefit, we demonstrate its advantages in semi-supervised rotation regression, where the pseudo labels are noisy.
Patch-based 3D Natural Scene Generation from a Single Example
We target a 3D generative model for general natural scenes that are typically unique and intricate.
Learning Controllable 3D Diffusion Models from Single-view Images
To address these challenges, We present Control3Diff, a 3D diffusion model that combines the strengths of diffusion models and 3D GANs for versatile, controllable 3D-aware image synthesis for single-view datasets.
Delving into Discrete Normalizing Flows on SO(3) Manifold for Probabilistic Rotation Modeling
Normalizing flows (NFs) provide a powerful tool to construct an expressive distribution by a sequence of trackable transformations of a base distribution and form a probabilistic model of underlying data.
A Laplace-inspired Distribution on SO(3) for Probabilistic Rotation Estimation
Rotation Laplace distribution is robust to the disturbance of outliers and enforces much gradient to the low-error region, resulting in a better convergence.
ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters
Our framework can learn a rich and flexible latent representation of skills and a skill-conditioned generative control policy from a diverse set of unorganized motion sequences, which enables the generation of realistic human behaviors by sampling in the latent space and allows high-level control policies to reuse the learned skills to accomplish a variety of downstream tasks.
Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings
We present a novel co-speech gesture synthesis method that achieves convincing results both on the rhythm and semantics.
Ranked #1 on
Gesture Generation
on TED Gesture Dataset
SinGRAV: Learning a Generative Radiance Volume from a Single Natural Scene
We present a 3D generative model for general natural scenes.
Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors
The learned representation can be used to synthesize novel view images of an arbitrary person from a sparse set of cameras, and further animate them with the user's pose control.
Visual Localization via Few-Shot Scene Region Classification
Visual (re)localization addresses the problem of estimating the 6-DoF (Degree of Freedom) camera pose of a query image captured in a known scene, which is a key building block of many computer vision and robotics applications.
Multi-Robot Active Mapping via Neural Bipartite Graph Matching
Previous approaches either choose the frontier as the goal position via a myopic solution that hinders the time efficiency, or maximize the long-term value via reinforcement learning to directly regress the goal position, but does not guarantee the complete map construction.
FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering
Inspired by the popular semi-supervised approach, FixMatch, we propose to leverage pseudo label filtering to facilitate the information flow from labeled data to unlabeled data in a teacher-student mutual learning framework.
Self-Conditioned Generative Adversarial Networks for Image Editing
We compare our models to a wide range of latent editing methods, and show that by alleviating the bias they achieve finer semantic control and better identity preservation through a wider range of transformations.
Projective Manifold Gradient Layer for Deep Rotation Regression
Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem.
Unsupervised Co-part Segmentation through Assembly
Co-part segmentation is an important problem in computer vision for its rich applications.
MoCo-Flow: Neural Motion Consensus Flow for Dynamic Humans in Stationary Monocular Cameras
In this paper, we introduce Neural Motion Consensus Flow (MoCo-Flow), a representation that models dynamic humans in stationary monocular cameras using a 4D continuous time-variant function.
Neural Implicit 3D Shapes from Single Images with Spatial Patterns
However, the image features for describing 3D point samplings of implicit functions are less effective when significant variations of occlusions, views, and appearances exist from the image.
Learning Skeletal Articulations with Neural Blend Shapes
Furthermore, we propose neural blend shapes--a set of corrective pose-dependent shapes which improve the deformation quality in the joint regions in order to address the notorious artifacts resulting from standard rigging and skinning.
CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds
For the first time, we propose a unified framework that can handle 9DoF pose tracking for novel rigid object instances as well as per-part pose tracking for articulated objects from known categories.
Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments
Localizing the camera in a known indoor environment is a key building block for scene mapping, robot navigation, AR, etc.
Towards Accurate Active Camera Localization
These approaches localize the camera in the discrete pose space and are agnostic to the localization-driven scene property, which restricts the camera pose accuracy in the coarse scale.
Implicit Multidimensional Projection of Local Subspaces
We propose a visualization method to understand the effect of multidimensional projection on local subspaces, using implicit function differentiation.
DO-Conv: Depthwise Over-parameterized Convolutional Layer
Moreover, in the inference phase, the depthwise convolution is folded into the conventional convolution, reducing the computation to be exactly equivalent to that of a convolutional layer without over-parameterization.
MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency
We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video. While previous methods rely on either rigging or inverse kinematics (IK) to associate a consistent skeleton with temporally coherent joint rotations, our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation.
Towards a Neural Graphics Pipeline for Controllable Image Generation
NGP decomposes the image into a set of interpretable appearance feature maps, uncovering direct control handles for controllable image generation.
Generative 3D Part Assembly via Dynamic Graph Learning
Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.
Unpaired Motion Style Transfer from Video to Animation
In this paper, we present a novel data-driven framework for motion style transfer, which learns from an unpaired collection of motions with style labels, and enables transferring motion styles not observed during training.
Skeleton-Aware Networks for Deep Motion Retargeting
In other words, our operators form the building blocks of a new deep motion processing framework that embeds the motion into a common latent space, shared by a collection of homeomorphic skeletons.
IROS 2019 Lifelong Robotic Vision Challenge -- Lifelong Object Recognition Report
This report summarizes IROS 2019-Lifelong Robotic Vision Competition (Lifelong Object Recognition Challenge) with methods and results from the top $8$ finalists (out of over~$150$ teams).
Multimodal Shape Completion via Conditional Generative Adversarial Networks
Several deep learning methods have been proposed for completing partial data from shape acquisition setups, i. e., filling the regions that were missing in the shape.
Deep Reflection Prior
Reflections are very common phenomena in our daily photography, which distract people's attention from the scene behind the glass.
AutoRemover: Automatic Object Removal for Autonomous Driving Videos
To deal with shadows, we build up an autonomous driving shadow dataset and design a deep neural network to detect shadows automatically.
Decoupling Features and Coordinates for Few-shot RGB Relocalization
To address this issue, we approach camera relocalization with a decoupled solution where feature extraction, coordinate regression, and pose estimation are performed separately.
PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes
We introduce PQ-NET, a deep neural network which represents and generates 3D shapes via sequential part assembly.
DLGAN: Disentangling Label-Specific Fine-Grained Features for Image Manipulation
By mapping the discrete label-specific attribute features into a continuous prior distribution, we leverage the advantages of both discrete labels and reference images to achieve image manipulation in a hybrid fashion.
A General Decoupled Learning Framework for Parameterized Image Operators
To overcome this limitation, we propose a new decoupled learning algorithm to learn from the operator parameters to dynamically adjust the weights of a deep network for image operators, denoted as the base network.
Learning Character-Agnostic Motion for Motion Retargeting in 2D
In order to achieve our goal, we learn to extract, directly from a video, a high-level latent motion representation, which is invariant to the skeleton geometry and the camera view.
Unpaired Point Cloud Completion on Real Scans using Adversarial Training
As 3D scanning solutions become increasingly popular, several deep learning setups have been developed geared towards that task of scan completion, i. e., plausibly filling in regions there were missed in the raw scans.
PointCNN: Convolution On X-Transformed Points
We present a simple and general framework for feature learning from point cloud.
Ranked #5 on
3D Semantic Segmentation
on DALES
Super Diffusion for Salient Object Detection
Following this analysis, we propose super diffusion, a novel inclusive learning-based framework for salient object detection, which makes the optimum and robust performance by integrating a large pool of feature spaces, scales and even features originally computed for non-diffusion-based salient object detection.
Image Smoothing via Unsupervised Learning
Image smoothing represents a fundamental component of many disparate computer vision and graphics applications.
Deep Video-Based Performance Cloning
After training a deep generative network using a reference video capturing the appearance and dynamics of a target actor, we are able to generate videos where this actor reenacts other performances.
Neural Material: Learning Elastic Constitutive Material and Damping Models from Sparse Data
The challenge is that such data is sparse as it is consistently given only on part of the surface.
Graphics
SketchyScene: Richly-Annotated Scene Sketches
We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level.
GRAINS: Generative Recursive Autoencoders for INdoor Scenes
We present a generative neural network which enables us to generate plausible 3D indoor scenes in large quantities and varieties, easily and highly efficiently.
Graphics
Decouple Learning for Parameterized Image Operators
Many different deep networks have been used to approximate, accelerate or improve traditional image operators, such as image smoothing, super-resolution and denoising.
DifNet: Semantic Segmentation by Diffusion Networks
Deep Neural Networks (DNNs) have recently shown state of the art performance on semantic segmentation tasks, however, they still suffer from problems of poor boundary localization and spatial fragmented predictions.
Neural Best-Buddies: Sparse Cross-Domain Correspondence
Correspondence between images is a fundamental problem in computer vision, with a variety of graphics applications.
PointCNN: Convolution On $\mathcal{X}$-Transformed Points
The proposed method is a generalization of typical CNNs to feature learning from point clouds, thus we call it PointCNN.
Ranked #1 on
3D Instance Segmentation
on S3DIS
(mIoU metric)
Neuron-level Selective Context Aggregation for Scene Segmentation
Contextual information provides important cues for disambiguating visually similar pixels in scene segmentation.
A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing
This paper proposes a deep neural network structure that exploits edge information in addressing representative low-level vision tasks such as layer separation and image filtering.
Bundle Optimization for Multi-aspect Embedding
Understanding semantic similarity among images is the core of a wide range of computer vision applications.
Revisiting Deep Intrinsic Image Decompositions
While invaluable for many computer vision applications, decomposing a natural image into intrinsic reflectance and shading layers represents a challenging, underdetermined inverse problem.
A Holistic Approach for Data-Driven Object Cutout
We show that the resulting P-maps may be used to evaluate how likely a rectangle proposal is to contain an instance of the class, and further process good proposals to produce an accurate object cutout mask.
Synthesizing Training Images for Boosting Human 3D Pose Estimation
Human 3D pose estimation from a single image is a challenging task with numerous applications.
