Search Results for author:
Found 55 papers, 17 papers with code
SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting
A lifelike talking head requires synchronized coordination of subject identity, lip movements, facial expressions, and head poses.
RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks
Dual-arm robots play a crucial role in improving efficiency and flexibility in complex multitasking scenarios.
HF-VTON: High-Fidelity Virtual Try-On via Consistent Geometric and Semantic Alignment
Virtual try-on technology has become increasingly important in the fashion and retail industries, enabling the generation of high-fidelity garment images that adapt seamlessly to target human models.
DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations
To address this issue, we propose a new task -- multi-round dual-speaker interaction for 3D talking head generation -- which requires models to handle and generate both speaking and listening behaviors in continuous conversation.
BitHydra: Towards Bit-flip Inference Cost Attack against Large Language Models
Large language models (LLMs) have shown impressive capabilities across a wide range of applications, but their ever-increasing size and resource demands make them vulnerable to inference cost attacks, where attackers induce victim LLMs to generate the longest possible output content.
AsynFusion: Towards Asynchronous Latent Consistency Models for Decoupled Whole-Body Audio-Driven Avatars
Whole-body audio-driven avatar pose and expression generation is a critical task for creating lifelike digital humans and enhancing the capabilities of interactive virtual agents, with wide-ranging applications in virtual reality, digital entertainment, and remote communication.
TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks
Our work thus offers a practical pathway for developing more capable lightweight VLMs while introducing a fresh theoretical lens to better understand and address alignment bottlenecks in constrained multimodal systems.
Black-box Adversaries from Latent Space: Unnoticeable Attacks on Human Pose and Shape Estimation
These findings underscore the urgent need to address and mitigate security risks associated with digital human generation systems.
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
Currently, existing surveys on LLM safety primarily focus on specific stages of the LLM lifecycle, e. g., deployment phase or fine-tuning phase, lacking a comprehensive understanding of the entire "lifechain" of LLMs.
Unicorn: Text-Only Data Synthesis for Vision Language Model Training
By eliminating the dependency on real images while maintaining data quality and diversity, our framework offers a cost-effective and scalable solution for VLMs training.
STAMICS: Splat, Track And Map with Integrated Consistency and Semantics for Dense RGB-D SLAM
Simultaneous Localization and Mapping (SLAM) is a critical task in robotics, enabling systems to autonomously navigate and understand complex environments.
MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System
Retrieval-Augmented Generation (RAG), while serving as a viable complement to large language models (LLMs), often overlooks the crucial aspect of text chunking within its pipeline.
ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis
Audio-driven human gesture synthesis is a crucial task with broad applications in virtual avatars, human-computer interaction, and creative content generation.
DH-RAG: A Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue
Retrieval-Augmented Generation (RAG) systems have shown substantial benefits in applications such as question answering and multi-turn dialogue \citep{lewis2020retrieval}.
VarGes: Improving Variation in Co-Speech 3D Gesture Generation via StyleCLIPS
Generating expressive and diverse human gestures from audio is crucial in fields like human-computer interaction, virtual reality, and animation.
TinyLLaVA-Video: A Simple Framework of Small-scale Large Multimodal Models for Video Understanding
We present the TinyLLaVA-Video, a video understanding model with parameters not exceeding 4B that processes video sequences in a simple manner, without the need for complex architectures, supporting both fps sampling and uniform frame sampling.
JTD-UAV: MLLM-Enhanced Joint Tracking and Description Framework for Anti-UAV Systems
To benchmark this task, we introduce the TDUAV dataset, the largest dataset for joint UAV tracking and intent understanding, featuring 1, 328 challenging video sequences, over 163K annotated thermal frames, and 3K VQA pairs.
EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Removing unwanted concepts from large-scale text-to-image (T2I) diffusion models while maintaining their overall generative quality remains an open challenge.
MambaVO: Deep Visual Odometry Based on Sequential Matching Refinement and Training Smoothing
To address this challenge, this paper proposes MambaVO, which conducts robust initialization, Mamba-based sequential matching refinement, and smoothed training to enhance the matching quality and improve the pose estimation in deep visual odometry.
Dust to Tower: Coarse-to-Fine Photo-Realistic Scene Reconstruction from Sparse Uncalibrated Images
To refine the 3D model at novel viewpoints, we propose a Confidence Aware Depth Alignment (CADA) module to refine the coarse depth maps by aligning their confident parts with estimated depths by a Mono-depth model.
Score and Distribution Matching Policy: Advanced Accelerated Visuomotor Policies via Matched Distillation
Visual-motor policy learning has advanced with architectures like diffusion-based policies, known for modeling complex robotic trajectories.
Moderating the Generalization of Score-based Generative Model
To fill this gap, we first examine the current 'gold standard' in Machine Unlearning (MU), i. e., re-training the model after removing the undesirable training data, and find it does not work in SGMs.
CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Prediction
In this paper, we introduce Coarse-to-Fine AutoRegressive Policy (CARP), a novel paradigm for visuomotor policy learning that redefines the autoregressive action generation process as a coarse-to-fine, next-scale approach.
VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstruction
Specifically, VGG-Tex includes a Facial Attributes Encoding Module, a Geometry-Guided Texture Generator, and a Visibility-Enhanced Texture Completion Module.
EmoFace: Emotion-Content Disentangled Speech-Driven 3D Talking Face Animation
EmoFace employs a novel Mesh Attention mechanism to analyse and fuse the emotion features and content features.
Meta-Learning Empowered Meta-Face: Personalized Speaking Style Adaptation for Audio-Driven 3D Talking Face Animation
Audio-driven 3D face animation is increasingly vital in live streaming and augmented reality applications.
GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer
Speech-driven talking head generation is a critical yet challenging task with applications in augmented reality and virtual human modeling.
MLPHand: Real Time Multi-View 3D Hand Mesh Reconstruction via MLP Modeling
Multi-view hand mesh reconstruction is a critical task for applications in virtual reality and human-computer interaction, but it remains a formidable challenge.
A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing
Talking head synthesis, an advanced method for generating portrait videos from a still image driven by specific content, has garnered widespread attention in virtual reality, augmented reality and game production.
Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs
Recently, this success has been extended to 3D AIGC, with state-of-the-art methods generating textured 3D models from single images or text.
Ultraman: Single Image 3D Human Reconstruction with Ultra Speed and Detail
In this paper, we propose a new method called \emph{Ultraman} for fast reconstruction of textured 3D human models from a single image.
Ranked #2 on
Lifelike 3D Human Generation
on THuman2.0 Dataset
AS-FIBA: Adaptive Selective Frequency-Injection for Backdoor Attack on Deep Face Restoration
Extensive experiments demonstrate the efficacy of the degradation objective on state-of-the-art face restoration models.
Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning
3D medical image segmentation is a challenging task with crucial implications for disease diagnosis and treatment planning.
Adversarial Purification and Fine-tuning for Robust UDC Image Restoration
This study delves into the enhancement of Under-Display Camera (UDC) image restoration models, focusing on their robustness against adversarial attacks.
SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis
A lifelike talking head requires synchronized coordination of subject identity, lip movements, facial expressions, and head poses.
BeatDance: A Beat-Based Model-Agnostic Contrastive Learning Framework for Music-Dance Retrieval
Dance and music are closely related forms of expression, with mutual retrieval between dance videos and music being a fundamental task in various fields like education, art, and sports.
STDG: Semi-Teacher-Student Training Paradigram for Depth-guided One-stage Scene Graph Generation
Scene Graph Generation is a critical enabler of environmental comprehension for autonomous robotic systems.
Multi-dimensional Fusion and Consistency for Semi-supervised Medical Image Segmentation
In this paper, we introduce a novel semi-supervised learning framework tailored for medical image segmentation.
D-IF: Uncertainty-aware Human Digitization via Implicit Distribution Field
In this paper, we propose replacing the implicit value with an adaptive uncertainty distribution, to differentiate between points based on their distance to the surface.
Benchmarking Ultra-High-Definition Image Reflection Removal
To the best of our knowledge, these two datasets are the first largest-scale UHD datasets for SIRR.
DenseMP: Unsupervised Dense Pre-training for Few-shot Medical Image Segmentation
Few-shot medical image semantic segmentation is of paramount importance in the domain of medical image analysis.
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces
To enhance the visual accuracy of generated lip movement while reducing the dependence on labeled data, we propose a novel framework SelfTalk, by involving self-supervision in a cross-modals network system to learn 3D talking faces.
EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation
Specifically, we introduce the emotion disentangling encoder (EDE) to disentangle the emotion and content in the speech by cross-reconstructed speech signals with different emotion labels.
Robust Single Image Reflection Removal Against Adversarial Attacks
This paper addresses the problem of robust deep single-image reflection removal (SIRR) against adversarial attacks.
Ranked #4 on
Reflection Removal
on Real20
SHLE: Devices Tracking and Depth Filtering for Stereo-based Height Limit Estimation
In stage 1, a novel devices detection and tracking scheme is introduced, which accurately locate the height limit devices in the left or right image.
FuRPE: Learning Full-body Reconstruction from Part Experts
These novel elements of FuRPE not only serve to further refine the model but also to reduce potential biases that may arise from inaccuracies in pseudo labels, thereby optimizing the network's training process and enhancing the robustness of the model.
GIDP: Learning a Good Initialization and Inducing Descriptor Post-enhancing for Large-scale Place Recognition
Large-scale place recognition is a fundamental but challenging task, which plays an increasingly important role in autonomous driving and robotics.
Human Pose Driven Object Effects Recommendation
In this paper, we research the new topic of object effects recommendation in micro-video platforms, which is a challenging but important task for many practical applications such as advertisement insertion.
MonoSIM: Simulating Learning Behaviors of Heterogeneous Point Cloud Object Detectors for Monocular 3D Object Detection
The insight behind introducing MonoSIM is that we propose to simulate the feature learning behaviors of a point cloud based detector for monocular detector during the training period.
Reconstruction-Aware Prior Distillation for Semi-supervised Point Cloud Completion
Real-world sensors often produce incomplete, irregular, and noisy point clouds, making point cloud completion increasingly important.
Object Level Depth Reconstruction for Category Level 6D Object Pose Estimation From Monocular RGB Image
Recently, RGBD-based category-level 6D object pose estimation has achieved promising improvement in performance, however, the requirement of depth information prohibits broader applications.
ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation
ACR-Pose consists of a Reconstructor and a Discriminator.
RPR-Net: A Point Cloud-based Rotation-aware Large Scale Place Recognition Network
Third, we apply these kernels to previous point cloud features to generate new features, which is the well-known SO(3) mapping process.
Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview
Therefore, this study presents a comprehensive review of recent progress in object pose detection and tracking that belongs to the deep learning technical route.
SVT-Net: Super Light-Weight Sparse Voxel Transformer for Large Scale Place Recognition
Point cloud-based large scale place recognition is fundamental for many applications like Simultaneous Localization and Mapping (SLAM).
Ranked #2 on
3D Place Recognition
on Oxford RobotCar Dataset
