Search Results for author:
Found 49 papers, 19 papers with code
EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head
We present a novel approach for synthesizing 3D talking heads with controllable emotion, featuring enhanced lip synchronization and rendering quality.
Head360: Learning a Parametric 3D Full-Head for Free-View Synthesis in 360°
To the best of our knowledge, our model is the first parametric 3D full-head that achieves 360{\deg} free-view synthesis, image-based fitting, appearance editing, and animation within a single model.
HINER: Neural Representation for Hyperspectral Image
For downstream classification on compressed HSI, we theoretically demonstrate the task accuracy is not only related to the classification loss but also to the reconstruction fidelity through a first-order expansion of the accuracy degradation, and accordingly adapt the reconstruction by introducing Adaptive Spectral Weighting.
FINER++: Building a Family of Variable-periodic Functions for Activating Implicit Neural Representation
By initializing the bias of the neural network with different ranges, sub-functions with various frequencies in the variable-periodic function are selected for activation.
Towards the Spectral bias Alleviation by Normalizations in Coordinate Networks
This problem is caused by the pathological distribution of the neural tangent kernel's (NTK's) eigenvalues of coordinate networks.
Joint RGB-Spectral Decomposition Model Guided Image Enhancement in Mobile Photography
The integration of miniaturized spectrometers into mobile devices offers new avenues for image quality enhancement and facilitates novel downstream tasks.
Neural Poisson Solver: A Universal and Continuous Framework for Natural Signal Blending
To tackle this issue, we introduce the Neural Poisson Solver, a plug-and-play and universally applicable framework across different signal dimensions for blending visual signals represented by INRs.
Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer
In this work, we introduce Direct3D, a native 3D generative model scalable to in-the-wild input images, without requiring a multiview diffusion model or SDS optimization.
MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors
To validate the dataset, we propose an RGBD-P SMPL fitting method and also a monocular-video-based baseline framework, VP-MoCap, for human motion capture.
STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians
Recent progress in pre-trained diffusion models and 3D generation have spurred interest in 4D content creation.
Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion guidance in curernt human generative techniques.
Batch Normalization Alleviates the Spectral Bias in Coordinate Networks
Representing signals using coordinate networks dominates the area of inverse problems recently and is widely applied in various scientific computing tasks.
FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions
Implicit Neural Representation (INR), which utilizes a neural network to map coordinate inputs to corresponding attributes, is causing a revolution in the field of signal processing.
VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Prior
Audio-driven talking head generation has drawn much attention in recent years, and many efforts have been made in lip-sync, expressive facial expressions, natural head pose generation, and high video quality.
Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion
The multi-view 2. 5D diffusion directly models the structural distribution of 3D data, while still maintaining the strong generalization ability of the original 2D diffusion model, filling the gap between 2D diffusion-based and direct 3D diffusion-based methods for 3D content generation.
Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing
In this paper, we present a novel differentiable point-based rendering framework to achieve photo-realistic relighting.
RefConv: Re-parameterized Refocusing Convolution for Powerful ConvNets
We propose Re-parameterized Refocusing Convolution (RefConv) as a replacement for regular convolutional layers, which is a plug-and-play module to improve the performance without any inference costs.
Continuous 3D Myocardial Motion Tracking via Echocardiography
Myocardial motion tracking stands as an essential clinical tool in the prevention and detection of cardiovascular diseases (CVDs), the foremost cause of death globally.
Aperture Diffraction for Compact Snapshot Spectral Imaging
We demonstrate a compact, cost-effective snapshot spectral imaging system named Aperture Diffraction Imaging Spectrometer (ADIS), which consists only of an imaging lens with an ultra-thin orthogonal aperture mask and a mosaic filter sensor, requiring no additional physical footprint compared to common RGB cameras.
RHINO: Regularizing the Hash-based Implicit Neural Representation
This connection ensures a seamless backpropagation of gradients from the network's output back to the input coordinates, thereby enhancing regularization.
Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail
Drawing inspiration from voxel-based representations with the level of detail (LoD), we introduce a multi-scale tri-plane-based scene representation that is capable of capturing the LoD of the signed distance function (SDF) and the space radiance.
AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation
Unlike previous approaches that can only synthesize avatars based on simple text descriptions, our method enables the creation of personalized avatars from casually captured face or body images, while still supporting text-based model generation and editing.
High-Fidelity 3D Face Generation from Natural Language Descriptions
Synthesizing high-quality 3D face models from natural language descriptions is very valuable for many applications, including avatar creation, virtual reality, and telepresence.
NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination
Recent advances in implicit neural representation have demonstrated the ability to recover detailed geometry and material from multi-view images.
Disorder-invariant Implicit Neural Representation
However, the expressive power of INR is limited by the spectral bias in the network training.
RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs
We propose a robust and accurate non-parametric method for single-view 3D face reconstruction (SVFR).
EEG Opto-processor: epileptic seizure detection using diffractive photonic computing units
Here, we propose the EEG opto-processor based on diffractive photonic computing units (DPUs) to effectively process the extracranial and intracranial EEG signals and perform epileptic seizure detection.
DINER: Disorder-Invariant Implicit Neural Representation
Implicit neural representation (INR) characterizes the attributes of a signal as a function of corresponding coordinates which emerges as a sharp weapon for solving inverse problems.
Explore Spatio-temporal Aggregation for Insubstantial Object Detection: Benchmark Dataset and Baseline
We endeavor on a rarely explored task named Insubstantial Object Detection (IOD), which aims to localize the object with following characteristics: (1) amorphous shape with indistinct boundary; (2) similarity to surroundings; (3) absence in color.
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model
Although significant progress has been made to audio-driven talking face generation, existing methods either neglect facial emotion or cannot be applied to arbitrary subjects.
Event Transformer
The event camera's low power consumption and ability to capture microsecond brightness changes make it attractive for various computer vision tasks.
Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function
By fitting a 3D morphable model from multi-view images, the features of multiple images are extracted and aggregated in the mesh-attached UV space, which makes the implicit function more effective in recovering detailed facial shape.
MoFaNeRF: Morphable Facial Neural Radiance Field
To the best of our knowledge, our work is the first facial parametric model built upon a neural radiance field that can be used in fitting, generation and manipulation.
FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction
By training on FaceScape data, a novel algorithm is proposed to predict elaborate riggable 3D face models from a single image input.
Live Speech Portraits: Real-Time Photorealistic Talking-Head Animation
The first stage is a deep neural network that extracts deep audio features along with a manifold projection to project the features to the target person's speech space.
Detailed Avatar Recovery from Single Image
In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation.
End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation
We further design a one-to-many decoder pipeline to generate multiple predictions from the CSTR, including vector-based resampling, adaptive kernel-based resampling, compensation mode selection maps and texture enhancements, and combines them adaptively to achieve more accurate inter prediction.
Audio-Driven Emotional Video Portraits
In this work, we present Emotional Video Portraits (EVP), a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios.
Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems
Compared with traditional pedestrian detection, we find multispectral pedestrian detection suffers from modality imbalance problems which will hinder the optimization process of dual-modality network and depress the performance of detector.
Neural Video Coding using Multiscale Motion Compensation and Spatiotemporal Context Model
Over the past two decades, traditional block-based video coding has made remarkable progress and spawned a series of well-known standards such as MPEG-4, H. 264/AVC and H. 265/HEVC.
FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction
In this paper, we present a large-scale detailed 3D face dataset, FaceScape, and propose a novel algorithm that is able to predict elaborate riggable 3D face models from a single image input.
Neural Image Compression via Non-Local Attention Optimization and Improved Context Modeling
This paper proposes a novel Non-Local Attention optmization and Improved Context modeling-based image compression (NLAIC) algorithm, which is built on top of the deep nerual network (DNN)-based variational auto-encoder (VAE) structure.
Hyperspectral City V1.0 Dataset and Benchmark
This document introduces the background and the usage of the Hyperspectral City Dataset and the benchmark.
Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformation
This paper presents a novel framework to recover detailed human body shapes from a single image.
Non-local Attention Optimized Deep Image Compression
This paper proposes a novel Non-Local Attention Optimized Deep Image Compression (NLAIC) framework, which is built on top of the popular variational auto-encoder (VAE) structure.
Multispectral Image Intrinsic Decomposition via Subspace Constraint
In this paper, a new Multispectral Image Intrinsic Decomposition model (MIID) is presented to decompose the shading and reflectance from a single multispectral image.
View Extrapolation of Human Body from a Single Image
We study how to synthesize novel views of human body from a single image.
Multispectral Image Intrinsic Decomposition via Low Rank Constraint
In this paper, a Low Rank Multispectral Image Intrinsic Decomposition model (LRIID) is presented to decompose the shading and reflectance from a single multispectral image.
Blind Optical Aberration Correction by Exploring Geometric and Visual Priors
Furthermore, by investigating the visual artifacts of aberration degenerated images captured by consumer-level cameras, the non-uniform distribution of sharpness across color channels and the image lattice is exploited as visual priors, resulting in a novel strategy to utilize the guidance from the sharpest channel and local image regions to improve the overall performance and robustness.
