Search Results for author:
Found 38 papers, 12 papers with code
Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation
To learn the sewing pattern distribution in the latent space, we design a two-step training strategy to inject the multi-modal conditions, \ie, body shapes, text prompts, and garment sketches, into a diffusion model, ensuring the generated garments are body-suited and detail-controlled.
Revealing Directions for Text-guided 3D Face Editing
Experiments demonstrate the effectiveness and generalization of our Face Clan for various pre-trained GANs.
FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Model
In the facial animation transfer component, we propose a novel Expression-driven Multi-avatar Animator, which first maps expressive semantics to the facial control parameters of 3D avatars and then imposes perceptual constraints between the input and output images to maintain expression consistency.
Affective Behaviour Analysis via Progressive Learning
To advance this, the 7th Affective Behavior Analysis in-the-wild (ABAW) competition establishes two tracks: i. e., the Multi-task Learning (MTL) Challenge and the Compound Expression (CE) challenge based on Aff-Wild2 and C-EXPR-DB datasets.
HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects
Thus, we propose HIMO, a large-scale MoCap dataset of full-body human interacting with multiple objects, containing 3. 3K 4D HOI sequences and 4. 08M 3D HOI frames.
Affective Behaviour Analysis via Integrating Multi-Modal Knowledge
Affective Behavior Analysis aims to facilitate technology emotionally smart, creating a world where devices can understand and react to our emotions as humans do.
EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Priors
This limitation leads to the Janus problem where multi-faced 3D models are generated under the guidance of such diffusion models.
Benchmarking Audio Visual Segmentation for Long-Untrimmed Videos
Considering that pixel-level annotations are difficult to achieve in some complex scenes we also provide the bounding boxes to indicate the sounding regions.
Text-Guided 3D Face Synthesis - From Generation to Editing
In the editing stage we first employ a pre-trained diffusion model to update facial geometry or texture based on the texts.
Text-Guided 3D Face Synthesis -- From Generation to Editing
In the editing stage, we first employ a pre-trained diffusion model to update facial geometry or texture based on the texts.
High-Quality 3D Face Reconstruction with Affine Convolutional Networks
In our method, an affine transformation matrix is learned from the affine convolution layer for each spatial location of the feature maps.
CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Poses
Existing volumetric neural rendering techniques, such as Neural Radiance Fields (NeRF), face limitations in synthesizing high-quality novel views when the camera poses of input images are imperfect.
EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior
This limitation leads to the Janus problem, where multi-faced 3D models are generated under the guidance of such diffusion models.
BAVS: Bootstrapping Audio-Visual Segmentation by Integrating Foundation Knowledge
In a nutshell, our BAVS is designed to eliminate the interference of background noise or off-screen sounds in segmentation by establishing the audio-visual correspondences in an explicit manner.
Zero-shot Text-driven Physically Interpretable Face Editing
Our method can also be flexibly extended to real-time video face editing.
Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation
Insufficient synergy neglects the idea that a robust 3D representation should align with the joint vision-language space, rather than independently aligning with each modality.
Audio-Visual Segmentation by Exploring Cross-Modal Mutual Semantics
However, we observed that prior arts are prone to segment a certain salient object in a video regardless of the audio information.
EmotionGesture: Audio-Driven Diverse Emotional Co-Speech 3D Gesture Generation
In this work, we propose EmotionGesture, a novel framework for synthesizing vivid and diverse emotional co-speech 3D gestures from audio.
NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illumination
Inverse rendering methods aim to estimate geometry, materials and illumination from multi-view RGB images.
Ranked #2 on
Surface Normals Estimation
on Stanford-ORB
DyGait: Exploiting Dynamic Representations for High-performance Gait Recognition
This is the first framework on gait recognition that is designed to focus on the extraction of dynamic features.
Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglement
Considering the asymmetric gestures and motions of two hands, we introduce a Spatial-Residual Memory (SRM) module to model spatial interaction between the body and each hand by residual learning.
Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation
In our method, taking the power of large-scale pre-trained multi-modal CLIP and neural rendering, T2P searches both continuous facial parameters and discrete facial parameters in a unified framework.
Towards Unbiased Volume Rendering of Neural Implicit Surfaces With Geometry Priors
In this paper, we argue that this limited accuracy is due to the bias of their volume rendering strategies, especially when the viewing direction is close to be tangent to the surface.
Object-Goal Visual Navigation via Effective Exploration of Relations Among Historical Navigation States
In HiNL, we propose a History-aware State Estimation (HaSE) module to alleviate the impacts of dominant historical states on the current state estimation.
FlowFace: Semantic Flow-guided Shape-aware Face Swapping
Unlike most previous methods that focus on transferring the source inner facial features but neglect facial contours, our FlowFace can transfer both of them to a target face, thus leading to more realistic face swapping.
Evidence-based Match-status-Aware Gait Recognition for Out-of-Gallery Gait Identification
We measure the uncertainty and predict the match status of the recognition results, and thus determine whether the probe is an OOG query. To the best of our knowledge, our method is the first attempt to tackle OOG queries in gait recognition.
Facial Action Units Detection Aided by Global-Local Expression Embedding
We find that AUs and facial expressions are highly associated, and existing facial expression datasets often contain a large number of identities.
GaitGL: Learning Discriminative Global-Local Feature Representations for Gait Recognition
GFR extractor aims to extract contextual information, e. g., the relationship among various body parts, and the mask-based LFR extractor is presented to exploit the detailed posture changes of local regions.
NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction with Implicit Neural Representations
In this paper, we explore for the first time the possibility of using implicit neural representations for autonomous 3D scene reconstruction by addressing two key challenges: 1) seeking a criterion to measure the quality of the candidate viewpoints for the view planning based on the new representations, and 2) learning the criterion from data that can generalize to different scenes instead of a hand-crafting one.
GaitStrip: Gait Recognition via Effective Strip-based Feature Representations and Multi-Level Framework
ECM consists of the Spatial-Temporal feature extractor (ST), the Frame-Level feature extractor (FL) and SPB, and has two obvious advantages: First, each branch focuses on a specific representation, which can be used to improve the robustness of the network.
Learning Implicit Body Representations from Double Diffusion Based Neural Radiance Fields
In this paper, we present a novel double diffusion based neural radiance field, dubbed DD-NeRF, to reconstruct human body geometry and render the human body appearance in novel views from a sparse set of images.
One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning
Hence, we propose a novel one-shot talking face generation framework by exploring consistent correlations between audio and visual motions from a specific speaker and then transferring audio-driven motion fields to a reference image.
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion
As this keypoint based representation models the motions of facial regions, head, and backgrounds integrally, our method can better constrain the spatial and temporal consistency of the generated videos.
Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition
Automatic affective recognition has been an important research topic in human computer interaction (HCI) area.
Flow-Guided One-Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset
To synthesize high-definition videos, we build a large in-the-wild high-resolution audio-visual dataset and propose a novel flow-guided talking face generation framework.
Learning a Deep Motion Interpolation Network for Human Skeleton Animations
In this work, we propose a novel carefully designed deep learning framework, named deep motion interpolation network (DMIN), to learn human movement habits from a real dataset and then to perform the interpolation function specific for human motions.
Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generation
To be specific, our framework consists of a speaker-independent stage and a speaker-specific stage.
Multi-label Relation Modeling in Facial Action Units Detection
The proposed method consists of the data preprocessing, the feature extraction and the AU classification.
