Search Results for author:
Found 135 papers, 38 papers with code
TensoRF: Tensorial Radiance Fields
We demonstrate that applying traditional CP decomposition -- that factorizes tensors into rank-one components with compact vectors -- in our framework leads to improvements over vanilla NeRF.
Ranked #3 on
Novel View Synthesis
on X3D
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
We introduce a large-scale 3D shape understanding benchmark using data and annotation from ShapeNet 3D object database.
MotionGPT: Human Motion as a Foreign Language
Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language.
MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo
We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis.
SofGAN: A Portrait Image Generator with Dynamic Styling
To address this issue, we propose a SofGAN image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space.
A Neural Rendering Framework for Free-Viewpoint Relighting
We present a novel Relightable Neural Renderer (RNR) for simultaneous view synthesis and relighting using multi-view image inputs.
Photo-Realistic Facial Details Synthesis from Single Image
Our technique employs expression analysis for proxy face geometry generation and combines supervised and unsupervised learning for facial detail synthesis.
Executing your Commands via Motion Diffusion in Latent Space
We study a challenging task, conditional human motion generation, which produces plausible human motion sequences according to various conditional inputs, such as action classes or textual descriptors.
Ranked #2 on
Motion Synthesis
on HumanAct12
Human Performance Modeling and Rendering via Neural Animated Mesh
In this paper, we present a comprehensive neural approach for high-quality reconstruction, compression, and rendering of human performances from dense multi-view videos.
Editable Free-viewpoint Video Using a Layered Neural Representation
Such layered representation supports fully perception and realistic manipulation of the dynamic scene whilst still supporting a free viewing experience in a wide range.
NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions
The spatial positions of their neural features are fixed on grid nodes and cannot well adapt to target signals.
GNeRF: GAN-based Neural Radiance Field without Posed Camera
We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field (NeRF) reconstruction for the complex scenarios with unknown and even randomly initialized camera poses.
Artemis: Articulated Neural Pets with Appearance and Motion synthesis
Our ARTEMIS enables interactive motion control, real-time animation, and photo-realistic rendering of furry animals.
Spatial-Angular Interaction for Light Field Image Super-Resolution
Specifically, spatial and angular features are first separately extracted from input LFs, and then repetitively interacted to progressively incorporate spatial and angular information.
InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions
Then, we propose a novel representation for motion input in our interaction diffusion model, which explicitly formulates the global relations between the two performers in the world frame.
Ranked #3 on
Motion Synthesis
on InterHuman
A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signals
One of the biggest challenges to this task is the one-to-many mapping from sparse observations to dense full-body motions, which endowed inherent ambiguities.
TightCap: 3D Human Shape Capture with Clothing Tightness Field
In this paper, we present TightCap, a data-driven scheme to capture both the human shape and dressed garments accurately with only a single 3D human scan, which enables numerous applications such as virtual try-on, biometrics and body evaluation.
SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos
In this paper, we propose SportsCap -- the first approach for simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input.
PREF: Phasorial Embedding Fields for Compact Neural Representations
We present an efficient frequency-based neural representation termed PREF: a shallow MLP augmented with a phasor volume that covers significant border spectra than previous Fourier feature mapping or Positional Encoding.
Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator
Text-to-video is a rapidly growing research area that aims to generate a semantic, identical, and temporal coherence sequence of frames that accurately align with the input text prompt.
HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR
We propose Human-centered 4D Scene Capture (HSC4D) to accurately and efficiently create a dynamic digital world, containing large-scale indoor-outdoor scenes, diverse human motions, and rich interactions between humans and environments.
ScalableMap: Scalable Map Learning for Online Long-Range Vectorized HD Map Construction
We extract more accurate bird's eye view (BEV) features guided by their linear structure, and then propose a hierarchical sparse map representation to further leverage the scalability of vectorized map elements and design a progressive decoding mechanism and a supervision strategy based on this representation.
An Arbitrary Scale Super-Resolution Approach for 3D MR Images via Implicit Neural Representation
In the ArSSR model, the reconstruction of HR images with different up-scaling rates is defined as learning a continuous implicit voxel function from the observed LR images.
AutoSweep: Recovering 3D Editable Objectsfrom a Single Photograph
We base our work on the assumption that most human-made objects are constituted by parts and these parts can be well represented by generalized primitives.
OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
A deep neural network of multiple nonlinear layers forms a large function space, which can easily lead to overfitting when it encounters small-sample data.
IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation
This paper presents an inverse kinematic optimization layer (IKOL) for 3D human pose and shape estimation that leverages the strength of both optimization- and regression-based methods within an end-to-end framework.
Ranked #25 on
3D Human Pose Estimation
on 3DPW
Neural Video Portrait Relighting in Real-time via Consistency Modeling
In this paper, we propose a neural approach for real-time, high-quality and coherent video portrait relighting, which jointly models the semantic, temporal and lighting consistency using a new dynamic OLAT dataset.
Non-line-of-Sight Imaging via Neural Transient Fields
We present a neural modeling framework for Non-Line-of-Sight (NLOS) imaging.
Learning Semantics-aware Distance Map with Semantics Layering Network for Amodal Instance Segmentation
Specifically, we first introduce a new representation, namely a semantics-aware distance map (sem-dist map), to serve as our target for amodal segmentation instead of the commonly used masks and heatmaps.
PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging
Hand modeling is critical for immersive VR/AR, action understanding, or human healthcare.
Human-centric Scene Understanding for 3D Large-scale Scenarios
Human-centric scene understanding is significant for real-world applications, but it is extremely challenging due to the existence of diverse human poses and actions, complex human-environment interactions, severe occlusions in crowds, etc.
Unsupervised Polychromatic Neural Representation for CT Metal Artifact Reduction
In this work, we present a novel Polychromatic neural representation (Polyner) to tackle the challenging problem of CT imaging when metallic implants exist within the human body.
Convolutional Neural Opacity Radiance Fields
In this paper, we propose a novel scheme to generate opacity radiance fields with a convolutional neural renderer for fuzzy objects, which is the first to combine both explicit opacity supervision and convolutional mechanism into the neural radiance field framework so as to enable high-quality appearance and global consistent alpha mattes generation in arbitrary novel views.
BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamics
We further provide a strong baseline method, BOTH2Hands, for the novel task: generating vivid two-hand motions from both implicit body dynamics and explicit text prompts.
Self-Supervised Coordinate Projection Network for Sparse-View Computed Tomography
Compared with recent related works that solve similar problems using implicit neural representation network (INR), our essential contribution is an effective and simple re-projection strategy that pushes the tomography image reconstruction quality over supervised deep learning CT reconstruction works.
Personalized Saliency and its Prediction
Specifically, we propose to decompose a personalized saliency map (referred to as PSM) into a universal saliency map (referred to as USM) predictable by existing saliency detection models and a new discrepancy map across users that characterizes personalized saliency.
Deep Coarse-to-fine Dense Light Field Reconstruction with Flexible Sampling and Geometry-aware Fusion
Specifically, the coarse sub-aperture image (SAI) synthesis module first explores the scene geometry from an unstructured sparsely-sampled LF and leverages it to independently synthesize novel SAIs, in which a confidence-based blending strategy is proposed to fuse the information from different input SAIs, giving an intermediate densely-sampled LF.
Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses
To the best of our knowledge, this is the first end-to-end deep learning method for reconstructing a high-resolution LF image with a hybrid input.
Semantic See-Through Rendering on Light Fields
We present a novel semantic light field (LF) refocusing technique that can achieve unprecedented see-through quality.
Deep Eyes: Binocular Depth-from-Focus on Focal Stack Pairs
We then construct three individual networks: a Focus-Net to extract depth from a single focal stack, a EDoF-Net to obtain the extended depth of field (EDoF) image from the focal stack, and a Stereo-Net to conduct stereo matching.
Robust 3D Human Motion Reconstruction Via Dynamic Template Construction
In multi-view human body capture systems, the recovered 3D geometry or even the acquired imagery data can be heavily corrupted due to occlusions, noise, limited field of- view, etc.
A Learning-based Framework for Hybrid Depth-from-Defocus and Stereo Matching
Depth from defocus (DfD) and stereo matching are two most studied passive depth sensing schemes.
Sparse Photometric 3D Face Reconstruction Guided by Morphable Models
We present a novel 3D face reconstruction technique that leverages sparse photometric stereo (PS) and latest advances on face registration/modeling from a single image.
Robust Guided Image Filtering
In this paper, we propose a general framework for Robust Guided Image Filtering (RGIF), which contains a data term and a smoothness term, to solve the two issues mentioned above.
Occlusion-Model Guided Anti-Occlusion Depth Estimation in Light Field
Occlusion is one of the most challenging problems in depth estimation.
Scene-adaptive Coded Apertures Imaging
Firstly, we capture the scene with a pinhole and analyze the scene content to determine primary edge orientations.
Robust High Quality Image Guided Depth Upsampling
A popular solution is upsampling the obtained noisy low resolution depth map with the guidance of the companion high resolution color image.
Automatic Layer Separation using Light Field Imaging
We propose a novel approach that jointly removes reflection or translucent layer from a scene and estimates scene depth.
Resolving Scale Ambiguity Via XSlit Aspect Ratio Analysis
We first conduct a comprehensive analysis to characterize DDAR, infer object depth from its AR, and model recoverable depth range, sensitivity, and error.
A Generic Multi-Projection-Center Model and Calibration Method for Light Field Cameras
Based on the MPC model and projective transformation, we propose a calibration algorithm to verify our light field camera model.
4D Human Body Correspondences from Panoramic Depth Maps
The availability of affordable 3D full body reconstruction systems has given rise to free-viewpoint video (FVV) of human shapes.
Deep Surface Light Fields
A surface light field represents the radiance of rays originating from any points on the surface in any directions.
Focus Manipulation Detection via Photometric Histogram Analysis
With the rise of misinformation spread via social media channels, enabled by the increasing automation and realism of image manipulation tools, image forensics is an increasingly relevant problem.
Automatic 3D Indoor Scene Modeling From Single Panorama
The recovered layout is then used to guide shape estimation of the remaining objects using their normal information.
Gaze Prediction in Dynamic 360° Immersive Videos
This paper explores gaze prediction in dynamic $360^circ$ immersive videos, emph{i. e.}, based on the history scan path and VR contents, we predict where a viewer will look at an upcoming time.
Saliency Detection in 360° Videos
Considering that the 360° videos are usually stored with equirectangular panorama, we propose to implement the spherical convolution on panorama by stretching and rotating the kernel based on the location of patch to be convolved.
Learning to Dodge A Bullet: Concyclic View Morphing via Deep Learning
The bullet-time effect, presented in feature film ``The Matrix", has been widely adopted in feature films and TV commercials to create an amazing stopping-time illusion.
Manhattan Scene Understanding via XSlit Imaging
Specifically, we prove that parallel 3D lines map to 2D curves in an XSlit image and they converge at an XSlit Vanishing Point (XVP).
Reconstructing Gas Flows Using Light-Path Approximation
By observing the LF-probe through the gas flow, we acquire a dense set of ray-ray correspondences and then reconstruct their light paths.
Fast Patch-Based Denoising Using Approximated Patch Geodesic Paths
PatchGPs treat image patches as nodes and patch differences as edge weights for computing the shortest (geodesic) paths.
Light Field Stereo Matching Using Bilateral Statistics of Surface Cameras
Our bilateral consistency metric is used to indicate the probability of occlusions by analyzing the SCams.
Saliency Detection on Light Field
Existing saliency detection approaches use images as inputs and are sensitive to foreground/background similarities, complex background textures, and occlusions.
Curvilinear Structure Tracking by Low Rank Tensor Approximation with Model Propagation
Robust tracking of deformable object like catheter or vascular structures in X-ray images is an important technique used in image guided medical interventions for effective motion compensation and dynamic multi-modality image fusion.
Aliasing Detection and Reduction in Plenoptic Imaging
When using plenoptic camera for digital refocusing, angular undersampling can cause severe (angular) aliasing artifacts.
Image Pre-compensation: Balancing Contrast and Ringing
In particular, we show that linear tone mapping eliminates ringing but incurs severe contrast loss, while non-linear tone mapping functions such as Gamma curves slightly enhances contrast but introduces ringing.
Ambient Occlusion via Compressive Visibility Estimation
This enables a sparsity-prior based solution for iteratively recovering the surface normal, the surface albedo, and the visibility function from a small number of images.
A Weighted Sparse Coding Framework for Saliency Detection
In this paper, we present a unified saliency detection framework for handling heterogenous types of input data.
Rotational Crossed-Slit Light Field
In this paper, we present a novel LF sampling scheme by exploiting a special non-centric camera called the crossed-slit or XSlit camera.
Image Splicing Detection via Camera Response Function Analysis
Recent advances on image manipulation techniques have made image forgery detection increasingly more challenging.
Depth Recovery From Light Field Using Focal Stack Symmetry
The other is a data consistency measure based on analysis-by-synthesis, i. e., the difference between the synthesized focal stack given the hypothesized depth map and that from the LF.
Catadioptric HyperSpectral Light Field Imaging
In this paper, we present a single camera hyperspectral light field imaging solution that we call Snapshot Plenoptic Imager (SPI).
Ray Space Features for Plenoptic Structure-From-Motion
In this paper, we explore using light fields captured by plenoptic cameras or camera arrays as inputs.
Hair Segmentation on Time-of-Flight RGBD Images
Robust segmentation of hair from portrait images remains challenging: hair does not conform to a uniform shape, style or even color; dark hair in particular lacks features.
3D Face Reconstruction Using Color Photometric Stereo with Uncalibrated Near Point Lights
We avoid the need for spatial constancy of albedo; instead, we use a new measure for albedo similarity that is based on the albedo norm profile.
Generic Multiview Visual Tracking
A natural solution to these challenges is to use multiple cameras with multiview inputs, though existing systems are mostly limited to specific targets (e. g. human), static cameras, and/or camera calibration.
Non-Lambertian Surface Shape and Reflectance Reconstruction Using Concentric Multi-Spectral Light Field
In this paper, we introduce a novel concentric multi-spectral light field (CMSLF) design that is able to recover the shape and reflectance of surfaces with arbitrary material in one shot.
PIV-Based 3D Fluid Flow Reconstruction Using Light Field Camera
Particle Imaging Velocimetry (PIV) estimates the flow of fluid by analyzing the motion of injected particles.
LGNN: A Context-aware Line Segment Detector
We present a novel real-time line segment detection scheme called Line Graph Neural Network (LGNN).
3D Fluid Flow Reconstruction Using Compact Light Field PIV
In this paper, we present a PIV solution that uses a compact lenslet-based light field camera to track dense particles floating in the fluid and reconstruct the 3D fluid flow.
MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging
In this paper, we propose MirrorNeRF - a one-shot neural portrait free-viewpoint rendering approach using a catadioptric imaging system with multiple sphere mirrors and a single high-resolution digital camera, which is the first to combine neural radiance field with catadioptric imaging so as to enable one-shot photo-realistic human portrait reconstruction and rendering, in a low-cost and casual capture setting.
IREM: High-Resolution Magnetic Resonance (MR) Image Reconstruction via Implicit Neural Representation
For collecting high-quality high-resolution (HR) MR image, we propose a novel image reconstruction network named IREM, which is trained on multiple low-resolution (LR) MR images and achieve an arbitrary up-sampling rate for HR image reconstruction.
Few-shot Neural Human Performance Rendering from Sparse RGBD Videos
To fill this gap, in this paper we propose a few-shot neural human rendering approach (FNHR) from only sparse RGBD inputs, which exploits the temporal and spatial redundancy to generate photo-realistic free-view output of human activities.
Neural Relighting and Expression Transfer On Video Portraits
We present a neural relighting and expression transfer technique to transfer the facial expressions from a source performer to a portrait video of a target performer while enabling dynamic relighting.
Neural Free-Viewpoint Performance Rendering under Complex Human-object Interactions
In this paper, we propose a neural human performance capture and rendering system to generate both high-quality geometry and photo-realistic texture of both human and objects under challenging interaction scenarios in arbitrary novel views, from only sparse RGB streams.
iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering
To fill this gap, in this paper we propose a neural interactive bullet-time generator (iButter) for photo-realistic human free-viewpoint rendering from dense RGB streams, which enables flexible and interactive design for human bullet-time visual effects.
Light Field-Based Underwater 3D Reconstruction Via Angular Resampling
Recovering 3D geometry of underwater scenes is challenging because of non-linear refraction of light at the water-air interface caused by the camera housing.
ProcK: Machine Learning for Knowledge-Intensive Processes
We present a novel methodology to build powerful predictive process models.
HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs
The raw HumanNeRF can already produce reasonable rendering on sparse video inputs of unseen subjects and camera settings.
Onsite Non-Line-of-Sight Imaging via Online Calibrations
There has been an increasing interest in deploying non-line-of-sight (NLOS) imaging systems for recovering objects behind an obstacle.
NIMBLE: A Non-rigid Hand Model with Bones and Muscles
Emerging Metaverse applications demand reliable, accurate, and photorealistic reproductions of human hands to perform sophisticated operations as if in the physical world.
Video-driven Neural Physically-based Facial Asset for Production
In this paper, we present a new learning-based, video-driven approach for generating dynamic facial geometries with high-quality physically-based assets.
NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing
We further develop a hybrid neural-rasterization rendering framework to support consumer-level VR headsets so that the aforementioned volumetric video viewing and editing, for the first time, can be conducted immersively in virtual 3D space.
Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time
In this paper, we present a novel Fourier PlenOctree (FPO) technique to tackle efficient neural modeling and real-time rendering of dynamic scenes captured under the free-view video (FVV) setting.
Disentangling Light Fields for Super-Resolution and Disparity Estimation
In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing.
NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions
In this paper, we propose NeuralHOFusion, a neural approach for volumetric human-object capture and rendering using sparse consumer RGBD sensors.
NeReF: Neural Refractive Field for Fluid Surface Reconstruction and Implicit Representation
We present a novel neural refractive field(NeReF) to recover wavefront of transparent fluids by simultaneously estimating the surface position and normal of the fluid front.
HybridCap: Inertia-aid Monocular Capture of Challenging Human Motions
Monocular 3D motion capture (mocap) is beneficial to many applications.
Ranked #1 on
Pose Estimation
on 3DPW
LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Clouds
Quantitative and qualitative experiments show that our method outperforms the techniques based only on RGB images.
Ranked #3 on
3D Human Pose Estimation
on SLOPER4D
(using extra training data)
LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors
We propose a multi-sensor fusion method for capturing challenging 3D human motions with accurate consecutive local poses and global trajectories in large-scale scenarios, only using single LiDAR and 4 IMUs, which are set up conveniently and worn lightly.
NARRATE: A Normal Assisted Free-View Portrait Stylizer
We extend Total Relighting to fix this problem by unifying its multi-view input normal maps with the physical face model.
Mutual Adaptive Reasoning for Monocular 3D Multi-Person Pose Estimation
This method first uses 2. 5D pose and geometry information to infer camera-centric root depths in a forward pass, and then exploits the root depths to further improve representation learning of 2. 5D pose estimation in a backward pass.
Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects
Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e. g., chairs) with large topological variance.
SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator
Named after the fossils of one of the oldest known human ancestors, our LUCY dataset contains high-quality Computed Tomography (CT) scans of the complete human head before and after orthognathic surgeries, critical for evaluating surgery results.
Joint Rigid Motion Correction and Sparse-View CT via Self-Calibrating Neural Field
NeRF-based SVCT methods represent the desired CT image as a continuous function of spatial coordinates and train a Multi-Layer Perceptron (MLP) to learn the function by minimizing loss on the SV sinogram.
LiCamGait: Gait Recognition in the Wild by Using LiDAR and Camera Multi-modal Visual Sensors
LiDAR can capture accurate depth information in large-scale scenarios without the effect of light conditions, and the captured point cloud contains gait-related 3D geometric properties and dynamic motion characteristics.
Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDAR
Motivated by this, we propose a monocular camera and single LiDAR-based method for 3D multi-person pose estimation in large-scale scenes, which is easy to deploy and insensitive to light.
Relightable Neural Human Assets from Multi-view Gradient Illuminations
UltraStage will be publicly available to the community to stimulate significant future developments in various human modeling and rendering tasks.
NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions
We construct a dense multi-view dome to acquire a complex human object interaction dataset, named HODome, that consists of $\sim$75M frames on 10 subjects interacting with 23 objects.
NEPHELE: A Neural Platform for Highly Realistic Cloud Radiance Rendering
Using such a cloud platform compatible with neural rendering, we further showcase the capabilities of our cloud radiance rendering through a series of applications, ranging from cloud VR/AR rendering.
CryoFormer: Continuous Heterogeneous Cryo-EM Reconstruction using Transformer-based Neural Representations
Cryo-electron microscopy (cryo-EM) allows for the high-resolution reconstruction of 3D structures of proteins and other biomolecules.
NeMF: Inverse Volume Rendering with Neural Microflake Field
Recovering the physical attributes of an object's appearance from its images captured under an unknown illumination is challenging yet essential for photo-realistic rendering.
High-resolution tomographic reconstruction of optical absorbance through scattering media using neural fields
Light scattering imposes a major obstacle for imaging objects seated deeply in turbid media, such as biological tissues and foggy air.
Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos
The success of the Neural Radiance Fields (NeRFs) for modeling and free-view rendering static objects has inspired numerous attempts on dynamic scenes.
Omni-Line-of-Sight Imaging for Holistic Shape Reconstruction
The core of our method is to put the object nearby diffuse walls and augment the LOS scan in the front view with the NLOS scans from the surrounding walls, which serve as virtual ``mirrors'' to trap lights toward the object.
Neural Impostor: Editing Neural Radiance Fields with Explicit Shape Manipulation
To tackle the problem of efficiently editing neural implicit fields, we introduce Neural Impostor, a hybrid representation incorporating an explicit tetrahedral mesh alongside a multigrid implicit field designated for each tetrahedron within the explicit mesh.
CoTDet: Affordance Knowledge Prompting for Task Driven Object Detection
Its challenge lies in object categories available for the task being too diverse to be limited to a closed set of object vocabulary for traditional object detection.
VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams
In this paper, we introduce VideoRF, the first approach to enable real-time streaming and rendering of dynamic radiance fields on mobile platforms.
GenEM: Physics-Informed Generative Cryo-Electron Microscopy
To address this, we introduce physics-informed generative cryo-electron microscopy (GenEM), which for the first time integrates physical-based cryo-EM simulation with a generative unpaired noise translation to generate physically correct synthetic cryo-EM datasets with realistic noises.
HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting
Then, we utilize a 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers to effectively balance the non-rigid prior and Gaussian updating.
HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models
We further present a strong baseline method HandDiffuse for the controllable motion generation of interacting hands using various controllers.
OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers
At the subsequent fine-tuning stage, we introduce motion ControlNet, which incorporates text prompts as conditioning information, through a trainable copy of the pre-trained model and the proposed novel Mixture-of-Controllers (MoC) block.
I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions
In this paper, we present I'm-HOI, a monocular scheme to faithfully capture the 3D motions of both the human and object in a novel setting: using a minimal amount of RGB camera and object-mounted Inertial Measurement Unit (IMU).
MVHuman: Tailoring 2D Diffusion with Multi-view Sampling For Realistic 3D Human Generation
Recent months have witnessed rapid progress in 3D generation based on diffusion models.
Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance
The synthesis of 3D facial animations from speech has garnered considerable attention.
DressCode: Autoregressively Sewing and Generating Garments from Text Guidance
For our framework, we first introduce SewingGPT, a GPT-based architecture integrating cross-attention with text-conditioned embedding to generate sewing patterns with text guidance.
Guidance with Spherical Gaussian Constraint for Conditional Diffusion
We theoretically show the existence of manifold deviation by establishing a certain lower bound for the estimation error of the loss guidance.
IMUSIC: IMU-based Facial Expression Capture
In this paper, we propose IMUSIC to fill the gap, a novel path for facial expression capture using purely IMU signals, significantly distant from previous visual solutions. The key design in our IMUSIC is a trilogy.
GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians
Hairstyle reflects culture and ethnicity at first glance.
RealDex: Towards Human-like Grasping for Robotic Dexterous Hand
In this paper, we introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns, enriched by multi-view and multimodal visual data.
LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment
For human-centric large-scale scenes, fine-grained modeling for 3D human global pose and shape is significant for scene understanding and can benefit many real-world applications.
THOR: Text to Human-Object Interaction Diffusion via Relation Intervention
This paper addresses new methodologies to deal with the challenging task of generating dynamic Human-Object Interactions from textual descriptions (Text2HOI).
Gaze-guided Hand-Object Interaction Synthesis: Benchmark and Method
Here, the object motion diffusion model generates sequences of object motions based on gaze conditions, while the hand motion diffusion model produces hand motions based on the generated object motion.
HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment
Humans naturally interact with both others and the surrounding multiple objects, engaging in various social activities.
LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives
We demonstrate that the collected LiDAR point cloud by the Polar device enhances a suite of 3D Gaussian splatting algorithms for garage scene modeling and rendering.
