Search Results for author:
Found 49 papers, 20 papers with code
LLM-driven Multimodal and Multi-Identity Listening Head Generation
Generating natural listener responses in conversational scenarios is crucial for creating engaging digital humans and avatars.
Towards Objectively Benchmarking Social Intelligence for Language Agents at Action Level
To gauge the significance of agent architecture, we implement a target-driven planning (TDP) module as an adjunct to the existing agent.
Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizer
In this paper, we propose a novel learning approach for feed-forward one-shot 4D head avatar synthesis.
Subobject-level Image Tokenization
Transformer-based vision models typically tokenize images into fixed-size square patches as input units, which lacks the adaptability to image content and overlooks the inherent pixel grouping structure.
Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations
Existing retrieval-based methods have made significant strides in maintaining long-term conversations.
AvatarGPT: All-in-One Framework for Motion Understanding Planning Generation and Beyond
Moreover for the first time AvatarGPT enables a principled approach by iterative traversal of the tasks within the closed-loop for unlimited long-motion synthesis.
From Good to Great: Improving Math Reasoning with Tool-Augmented Interleaf Prompting
IMP-TIP follows the ``From Good to Great" concept, collecting multiple potential solutions from both LLMs and their Tool-Augmented counterparts for the same math problem, and then selecting or re-generating the most accurate answer after cross-checking these solutions via tool-augmented interleaf prompting.
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance
Our proposed GSmoothFace model mainly consists of the Audio to Expression Prediction (A2EP) module and the Target Adaptive Face Translation (TAFT) module.
PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsigns
Unlike prior arts constrained by specific input types, our method allows flexible specification of style (text or image) and texture (full garment, cropped sections, or texture patches) conditions.
Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data
The key is to first learn a part-wise 4D generative model from monocular images via adversarial learning, to synthesize multi-view images of diverse identities and full motions as training data; then leverage a transformer-based animatable triplane reconstructor to learn 4D head reconstruction using the synthetic data.
AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agents
In this study, our goal is to create interactive avatar agents that can autonomously plan and animate nuanced facial movements realistically, from both visual and behavioral perspectives.
AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyond
AvatarGPT treats each task as one type of instruction fine-tuned on the shared LLM.
A Unified Framework for Multimodal, Multi-Part Human Motion Synthesis
The field has made significant progress in synthesizing realistic human motion driven by various modalities.
HAVE-FUN: Human Avatar Reconstruction from Few-Shot Unconstrained Images
As for human avatar reconstruction, contemporary techniques commonly necessitate the acquisition of costly data and struggle to achieve satisfactory results from a small number of casual images.
DialCoT Meets PPO: Decomposing and Exploring Reasoning Paths in Smaller Language Models
Chain-of-Thought (CoT) prompting has proven to be effective in enhancing the reasoning capabilities of Large Language Models (LLMs) with at least 100 billion parameters.
MDSC: Towards Evaluating the Style Consistency Between Music and Dance
We found that directly measuring the embedding distance between motion and music is not an optimal solution.
Controlling Character Motions without Observable Driving Source
A high-level prior model can be easily injected on top to generate unlimited long and diverse sequences.
Reinforced Disentanglement for Face Swapping without Skip Connection
The SOTA face swap models still suffer the problem of either target identity (i. e., shape) being leaked or the target non-identity attributes (i. e., background, hair) failing to be fully preserved in the final results.
Visual Instruction Tuning with Polite Flamingo
This side effect negatively impacts the model's ability to format responses appropriately -- for instance, its "politeness" -- due to the overly succinct and unformatted nature of raw annotations, resulting in reduced human preference.
LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streaming
Open-domain dialogue systems have made promising progress in recent years.
Hierarchical Verbalizer for Few-Shot Hierarchical Text Classification
Due to the complex label hierarchy and intensive labeling cost in practice, the hierarchical text classification (HTC) suffers a poor performance especially when low-resource or few-shot settings are considered.
ViTMatte: Boosting Image Matting with Pretrained Plain Vision Transformers
Recently, plain vision Transformers (ViTs) have shown impressive performance on various computer vision tasks, thanks to their strong modeling capacity and large-scale pretraining.
Ranked #2 on
Image Matting
on Distinctions-646
FashionTex: Controllable Virtual Try-on with Text and Texture
Virtual try-on attracts increasing research attention as a promising way for enhancing the user experience for online cloth shopping.
An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds
Due to the motion-centric nature, our method shows its impressive generalizability with limited training labels and provides good differentiability for end-to-end cycle training.
Mimic3D: Thriving 3D-Aware GANs via 3D-to-2D Imitation
Improving the photorealism via CNN-based 2D super-resolution can break the strict 3D consistency, while keeping the 3D consistency by learning high-resolution 3D representations for direct rendering often compromises image quality.
Orca: A Few-shot Benchmark for Chinese Conversational Machine Reading Comprehension
Thus, model's comprehension ability towards real scenarios are hard to evaluate reasonably.
Natural Response Generation for Chinese Reading Comprehension
To this end, we construct a new dataset called Penguin to promote the research of MRC, providing a training and test bed for natural response generation to real scenarios.
Chinese Reading Comprehension
Machine Reading Comprehension
+1
Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priors
In this paper, we introduce a simple and novel framework for one-shot audio-driven talking head generation.
UDE: A Unified Driving Engine for Human Motion Generation
Generating controllable and editable human motion sequences is a key challenge in 3D Avatar generation.
Ranked #2 on
Motion Synthesis
on AIST++
Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesis
We present a novel one-shot talking head synthesis method that achieves disentangled and fine-grained control over lip motion, eye gaze&blink, head pose, and emotional expression.
Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Image
We introduce a novel detail manifolds reconstructor to learn 3D-consistent fine details on the radiance manifolds from monocular images, and combine them with the coarse radiance manifolds for high-fidelity reconstruction.
Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video
We present HandAvatar, a novel representation for hand animation and rendering, which can generate smoothly compositional geometry and self-occlusion-aware texture.
Local-Adaptive Face Recognition via Graph-based Meta-Clustering and Regularized Adaptation
Leveraging the environment-specific local data after the deployment of the initial global model, LaFR aims at getting optimal performance by training local-adapted models automatically and un-supervisely, as opposed to fixing their initial global model.
Privacy-preserving Online AutoML for Domain-Specific Face Detection
Thanks to HyperFD, each local task (client) is able to effectively leverage the learning "experience" of previous tasks without uploading raw images to the platform; meanwhile, the meta-feature extractor is continuously learned to better trade off the bias and variance.
Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds
3D single object tracking (3D SOT) in LiDAR point clouds plays a crucial role in autonomous driving.
Ranked #1 on
Object Tracking
on KITTI
CRFace: Confidence Ranker for Model-Agnostic Face Detection Refinement
Our confidence ranker is model-agnostic, so we can augment the data by choosing the pairs from multiple face detectors during the training, and generalize to a wide range of face detectors during the testing.
JNR: Joint-based Neural Rig Representation for Compact 3D Face Modeling
In this paper, we introduce a novel approach to learn a 3D face model using a joint-based face rig and a neural skinning network.
Personalized Face Modeling for Improved Face Reconstruction and Motion Retargeting
Traditional methods for image-based 3D face reconstruction and facial motion retargeting fit a 3D morphable model (3DMM) to the face, which has limited modeling capacity and fail to generalize well to in-the-wild data.
Animating Face using Disentangled Audio Representations
All previous methods for audio-driven talking head generation assume the input audio to be clean with a neutral tone.
Joint Face Detection and Facial Motion Retargeting for Multiple Faces
Facial motion retargeting is an important problem in both computer graphics and vision, which involves capturing the performance of a human face and transferring it to another 3D character.
Real-time Burst Photo Selection Using a Light-Head Adversarial Network
The system is designed to run in the viewfinder mode and capture a burst sequence of frames before and after the shutter is pressed.
Personalized Exposure Control Using Adaptive Metering and Reinforcement Learning
We propose a reinforcement learning approach for real-time exposure control of a mobile camera that is personalizable.
Understanding and Predicting The Attractiveness of Human Action Shot
Selecting attractive photos from a human action shot sequence is quite challenging, because of the subjective nature of the "attractiveness", which is mainly a combined factor of human pose in action and the background.
Exposure: A White-Box Photo Post-Processing Framework
Retouching can significantly elevate the visual appeal of photos, but many casual photographers lack the expertise to do this well.
Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning
Cinemagraphs are a compelling way to convey dynamic aspects of a scene.
FC4: Fully Convolutional Color Constancy With Confidence-Weighted Pooling
However, the patch-based CNNs that exist for this problem are faced with the issue of estimation ambiguity, where a patch may contain insufficient information to establish a unique or even a limited possible range of illumination colors.
Harvesting Discriminative Meta Objects with Deep CNN Features for Scene Classification
Recent work on scene classification still makes use of generic CNN features in a rudimentary manner.
Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders
With the growing popularity of short-form video sharing platforms such as \em{Instagram} and \em{Vine}, there has been an increasing need for techniques that automatically extract highlights from video.
Automatic Photo Adjustment Using Deep Neural Networks
Many photographic styles rely on subtle adjustments that depend on the image content and even its semantics.
