Search Results for author:
Found 91 papers, 56 papers with code
Sparse-to-Dense Depth Completion Revisited: Sampling Strategy and Graph Construction
Depth completion is a widely studied problem of predicting a dense depth map from a sparse set of measurements and a single RGB image.
DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting
We also introduce a synthetic dataset to assess refocusing capabilities and the model's ability to learn precise lens parameters.
Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos
Existing methods either (1) assume sharp image inputs, failing to address the detail loss introduced by motion blur, or (2) mainly consider blur by camera movements, neglecting the human motion blur which is more common in animatable avatars.
Self-Distilled Depth Refinement with Noisy Poisson Fusion
Analyzing the fundamental reasons for these limitations, we model depth refinement as a noisy Poisson fusion problem with local inconsistency and edge deformation noises.
DreamMover: Leveraging the Prior of Diffusion Models for Image Interpolation with Large Motion
Due to the large motion, the intermediate semantic information may be absent in input images.
Training Matting Models without Alpha Labels
Inspired by the nonlocal principle in traditional image matting, we build a directional distance consistency loss (DDC loss) at each pixel neighborhood to constrain the alpha values conditioned on the input image.
Towards Robust Monocular Depth Estimation in Non-Lambertian Surfaces
In the field of monocular depth estimation (MDE), many models with excellent zero-shot performance in general scenes emerge recently.
iControl3D: An Interactive System for Controllable 3D Scene Generation
To this end, a 3D creator interface has been developed to provide users with fine-grained control over the creation process.
Exposure Completing for Temporally Consistent Neural High Dynamic Range Video Rendering
Combining the interpolated and given LDR frames, the complete set of exposure information is available at each time stamp.
FADE: A Task-Agnostic Upsampling Operator for Encoder-Decoder Architectures
The goal of this work is to develop a task-agnostic feature upsampling operator for dense prediction where the operator is required to facilitate not only region-sensitive tasks like semantic segmentation but also detail-sensitive tasks such as image matting.
Dynamic Neural Radiance Field From Defocused Monocular Video
We introduce layered Depth-of-Field (DoF) volume rendering to model the defocus blur and reconstruct a sharp NeRF supervised by defocused views.
Pseudo-Labeling by Multi-Policy Viewfinder Network for Image Cropping
This idea can be implemented in a pseudo-labeling way: producing pseudo labels for unlabeled data by a teacher model and training a student model with these pseudo labels.
WildAvatar: Web-scale In-the-wild Video Dataset for 3D Avatar Creation
However, their annotating requirements are impractical for real-world images or videos, posing challenges toward real-world applications on current avatar creation methods.
Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation
To leverage unlabeled data, previous semi-supervised 3D instance segmentation approaches have explored self-training frameworks, which rely on high-quality pseudo labels for consistency regularization.
MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo
We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) that can efficiently reconstruct unseen scenes.
Geometry-aware Reconstruction and Fusion-refined Rendering for Generalizable Neural Radiance Fields
We incorporate the above ACA, SVA, and CAF into a coarse-to-fine framework, termed Geometry-aware Reconstruction and Fusion-refined Rendering (GeFu).
3D Multi-frame Fusion for Video Stabilization
In this paper, we present RStab, a novel framework for video stabilization that integrates 3D multi-frame fusion through volume rendering.
Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations
To this end, we explore the feasibility of self-supervised motion prediction with only unlabeled LiDAR point clouds.
DyBluRF: Dynamic Neural Radiance Fields from Blurry Monocular Video
Experimental results on our dataset demonstrate that our method outperforms existing approaches in generating sharp novel views from motion-blurred inputs while maintaining spatial-temporal consistency of the scene.
CrossGLG: LLM Guides One-shot Skeleton-based 3D Action Recognition in a Cross-level Manner
Most existing one-shot skeleton-based action recognition focuses on raw low-level information (e. g., joint location), and may suffer from local information loss and low generalization ability.
S-DyRF: Reference-Based Stylized Radiance Fields for Dynamic Scenes
To address this limitation, we present S-DyRF, a reference-based spatio-temporal stylization method for dynamic neural radiance fields.
Unifying Automatic and Interactive Matting with Pretrained ViTs
Therefore we wonder whether we can alleviate the limitations of both settings while achieving unification to facilitate more convenient use.
Semi-Supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix
To improve the quality of pseudo labels, we propose a novel motion selection and re-generation module.
End-to-end Video Gaze Estimation via Capturing Head-face-eye Spatial-temporal Interaction Context
In this letter, we propose a new method, Multi-Clue Gaze (MCGaze), to facilitate video gaze estimation via capturing spatial-temporal interaction context among head, face, and eye in an end-to-end learning way, which has not been well concerned yet.
Ranked #1 on
Gaze Estimation
on Gaze360
When Epipolar Constraint Meets Non-local Operators in Multi-View Stereo
This constraint reduces the 2D search space into the epipolar line in stereo matching.
Ranked #4 on
3D Reconstruction
on DTU
Learning to Upsample by Learning to Sample
We present DySample, an ultra-lightweight and effective dynamic upsampler.
Point-Query Quadtree for Crowd Counting, Localization, and More
Such a querying process yields an intuitive, universal modeling of crowd as both the input and output are interpretable and steerable.
Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image
To animate the visual content, the feature point cloud is displaced based on the scene flow derived from motion estimation and the corresponding camera pose.
Diffusion-Augmented Depth Prediction with Sparse Annotations
We leverage the structural characteristics of diffusion model to enforce depth structures of depth models in a plug-and-play manner.
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World
We present the All-Seeing (AS) project: a large-scale data and model for recognizing and understanding everything in the open world.
Fast Full-frame Video Stabilization with Iterative Optimization
Video stabilization refers to the problem of transforming a shaky video into a visually pleasing one.
Constraining Depth Map Geometry for Multi-View Stereo: A Dual-Depth Approach with Saddle-shaped Depth Cells
Learning-based multi-view stereo (MVS) methods deal with predicting accurate depth maps to achieve an accurate and complete 3D representation.
Box-DETR: Understanding and Boxing Conditional Spatial Queries
In DAB-DETR, such queries are modulated by the so-called conditional linear projection at each decoder stage, aiming to search for positions of interest such as the four extremities of the box.
On Point Affiliation in Feature Upsampling
We introduce the notion of point affiliation into feature upsampling.
NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation
Our work serves as a solid baseline and data foundation for learning-based video depth estimation.
Ranked #21 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priors
To this end, we present a Defocus to Focus (D2F) framework to learn realistic bokeh rendering by fusing defocus priors with the all-in-focus image and by implementing radiance priors in layered fusion.
Learning Probabilistic Coordinate Fields for Robust Correspondences
We introduce Probabilistic Coordinate Fields (PCFs), a novel geometric-invariant coordinate representation for image correspondence problems.
A2B: Anchor to Barycentric Coordinate for Robust Correspondence
This approach offers a new perspective to alleviate the problem of repeated patterns and emphasizes the importance of choosing coordinate representations for feature correspondences.
Lens-to-lens bokeh effect transformation. NTIRE 2023 challenge report
We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge.
Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting
This task is typically addressed by extracting the features of query image and exemplars respectively and then matching their feature similarity, leading to an extract-then-match paradigm.
Ranked #4 on
Object Counting
on FSC147
Point-and-Shoot All-in-Focus Photo Synthesis from Smartphone Camera Pair
Benefiting from the multi-camera module in modern smartphones, we introduce a new task of AIF synthesis from main (wide) and ultra-wide cameras.
A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image
3D interacting hand pose estimation from a single RGB image is a challenging task, due to serious self-occlusion and inter-occlusion towards hands, confusing similar appearance patterns between 2 hands, ill-posed joint position mapping from 2D to 3D, etc.. To address these, we propose to extend A2J-the state-of-the-art depth-based 3D single hand pose estimation method-to RGB domain under interacting hand condition.
Ranked #7 on
Hand Pose Estimation
on NYU Hands
Real-time Multi-person Eyeblink Detection in the Wild for Untrimmed Video
Experiments on MPEblink verify the essential challenges of real-time multi-person eyeblink detection in the wild for untrimmed video.
Learning Second-Order Attentive Context for Efficient Correspondence Pruning
It is challenging because of the disorganized spatial distribution of numerous outliers, especially when putative correspondences are largely dominated by outliers.
3D Cinemagraphy from a Single Image
To animate the scene, we perform motion estimation and lift the 2D motion into the 3D scene flow.
Find Beauty in the Rare: Contrastive Composition Feature Clustering for Nontrivial Cropping Box Regression
Observing that similar composition patterns tend to be shared by the cropping boundaries annotated nearly, we argue to find the beauty of composition from the rare samples by clustering the samples with similar cropping boundary annotations, ie, similar composition patterns.
Matching Is Not Enough: A Two-Stage Framework for Category-Agnostic Pose Estimation
To calibrate the inaccurate matching results, we introduce a two-stage framework, where matched keypoints from the first stage are viewed as similarity-aware position proposals.
Ranked #5 on
2D Pose Estimation
on MP-100
Infusing Definiteness into Randomness: Rethinking Composition Styles for Deep Image Matting
Inspired by this, we introduce a novel composition style that binds the source and combined foregrounds in a definite triplet.
Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report
Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks.
SymmNeRF: Learning to Explore Symmetry Prior for Single-View View Synthesis
We study the problem of novel view synthesis of objects from a single image.
SAPA: Similarity-Aware Point Affiliation for Feature Upsampling
We introduce point affiliation into feature upsampling, a notion that describes the affiliation of each upsampled point to a semantic cluster formed by local decoder feature points with semantic similarity.
Ranked #5 on
Feature Upsampling
on ImageNet
DoF-NeRF: Depth-of-Field Meets Neural Radiance Fields
To mitigate this issue, we introduce DoF-NeRF, a novel neural rendering approach that can deal with shallow DoF inputs and can simulate DoF effect.
Less is More: Consistent Video Depth Estimation with Masked Frames Modeling
Temporal consistency is the key challenge of video depth estimation.
Design What You Desire: Icon Generation from Orthogonal Application and Theme Labels
Generative adversarial networks (GANs) have been trained to be professional artists able to create stunning artworks such as face generation and image style transfer.
FADE: Fusing the Assets of Decoder and Encoder for Task-Agnostic Upsampling
We consider the problem of task-agnostic feature upsampling in dense prediction where an upsampling operator is required to facilitate both region-sensitive tasks like semantic segmentation and detail-sensitive tasks such as image matting.
Robust Object Detection With Inaccurate Bounding Boxes
As the crowd-sourcing labeling process and the ambiguities of the objects may raise noisy bounding box annotations, the object detectors will suffer from the degenerated training data.
MPIB: An MPI-Based Bokeh Rendering Framework for Realistic Partial Occlusion Effects
Partial occlusion effects are a phenomenon that blurry objects near a camera are semi-transparent, resulting in partial appearance of occluded background.
3D Instances as 1D Kernels
The idea of instance kernel is inspired by recent success of dynamic convolutions in 2D/3D instance segmentation.
Ranked #2 on
3D Instance Segmentation
on S3DIS
(mCov metric)
BokehMe: When Neural Rendering Meets Classical Rendering
Based on this formulation, we implement the classical renderer by a scattering-based method and propose a two-stage neural renderer to fix the erroneous areas from the classical renderer.
Interior Attention-Aware Network for Infrared Small Target Detection
Motivated by the fact that pixels from targets or backgrounds are correlated to each other, we propose a coarse-to-fine interior attention-aware network (IAANet) for infrared small target detection.
Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting
In this work, we propose a similarity-aware CAC framework that jointly learns representation and similarity metric.
Ranked #4 on
Object Counting
on CARPK
Composing Photos Like a Photographer
To this end, we introduce the concept of the key composition map (KCM) to encode the composition rules.
Ranked #1 on
Image Cropping
on FLMS
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
While many solutions have been proposed for this task, they are usually very computationally expensive and thus are not applicable for on-device inference.
TransView: Inside, Outside, and Across the Cropping View Boundaries
We show that relation modeling between visual elements matters in cropping view recommendation.
On Efficient and Robust Metrics for RANSAC Hypotheses and 3D Rigid Registration
This paper focuses on developing efficient and robust evaluation metrics for RANSAC hypotheses to achieve accurate 3D rigid registration.
AIM 2020 Challenge on Rendering Realistic Bokeh
This paper reviews the second AIM realistic bokeh effect rendering challenge and provides the description of the proposed solutions and results.
3D Correspondence Grouping with Compatibility Features
We present a simple yet effective method for 3D correspondence grouping.
Weighing Counts: Sequential Crowd Counting by Reinforcement Learning
Inspired by scale weighing, we propose a novel 'counting scale' termed LibraNet where the count value is analogized by weight.
ECML: An Ensemble Cascade Metric Learning Mechanism towards Face Verification
Embedding RMML into the proposed ECML mechanism, our metric learning paradigm (EC-RMML) can run in the one-pass learning manner.
P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds
Specifically, we first sample seeds from the point clouds in template and search area respectively.
3DV: 3D Dynamic Voxel for Action Recognition in Depth Video
Each available 3DV voxel intrinsically involves 3D spatial and motion feature jointly.
Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction
To address these issues, we designed a public challenge (HANDS'19) to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set.
LRF-Net: Learning Local Reference Frames for 3D Local Shape Description and Matching
The local reference frame (LRF) acts as a critical role in 3D local shape description and matching.
From Open Set to Closed Set: Supervised Spatial Divide-and-Conquer for Object Counting
Visual counting, a task that aims to estimate the number of objects from an image/video, is an open-set problem by nature, i. e., the number of population can vary in [0, inf) in theory.
Rotation Invariant Point Cloud Classification: Where Local Geometry Meets Global Topology
To the best of our knowledge, this work is the first principled approach toward adaptively combining global and local information under the context of RI point cloud analysis.
Iterative Clustering with Game-Theoretic Matching for Robust Multi-consistency Correspondence
Matching corresponding features between two images is a fundamental task to computer vision with numerous applications in object recognition, robotics, and 3D reconstruction.
A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Image
For 3D hand and body pose estimation task in depth image, a novel anchor-based approach termed Anchor-to-Joint regression network (A2J) with the end-to-end learning ability is proposed.
Ranked #1 on
Hand Pose Estimation
on K2HPD
From Open Set to Closed Set: Counting Objects by Spatial Divide-and-Conquer
A dense region can always be divided until sub-region counts are within the previously observed closed set.
Ranked #3 on
Crowd Counting
on TRANCOS
Comparative evaluation of 2D feature correspondence selection algorithms
Correspondence selection aiming at seeking correct feature correspondences from raw feature matches is pivotal for a number of feature-matching-based tasks.
Learning to Fuse Local Geometric Features for 3D Rigid Data Matching
This paper presents a simple yet very effective data-driven approach to fuse both low-level and high-level local geometric features for 3D rigid data matching.
NM-Net: Mining Reliable Neighbors for Robust Feature Correspondences
Feature correspondence selection is pivotal to many feature-matching based tasks in computer vision.
Towards Real-time Eyeblink Detection in The Wild:Dataset,Theory and Practices
Effective and real-time eyeblink detection is of wide-range applications, such as deception detection, drive fatigue detection, face anti-spoofing, etc.
Towards Good Practices on Building Effective CNN Baseline Model for Person Re-identification
Person re-identification is indeed a challenging visual recognition task due to the critical issues of human pose variation, human body occlusion, camera view variation, etc.
Deep attention-based classification network for robust depth prediction
However, robust depth prediction suffers from two challenging problems: a) How to extract more discriminative features for different scenes (compared to a single scene)?
Action Recognition for Depth Video using Multi-view Dynamic Images
To better exploit three-dimensional (3D) characteristics, multi-view dynamic images are proposed.
Monocular Depth Estimation with Augmented Ordinal Depth Relationships
In this paper, we propose to improve the performance of metric depth estimation with relative depths collected from stereo movie videos using existing stereo matching algorithm.
Monocular Relative Depth Perception With Web Stereo Data Supervision
In this paper we study the problem of monocular relative depth perception in the wild.
Performance Evaluation of 3D Correspondence Grouping Algorithms
This paper presents a thorough evaluation of several widely-used 3D correspondence grouping algorithms, motived by their significance in vision tasks relying on correct feature correspondences.
When Unsupervised Domain Adaptation Meets Tensor Representations
Domain adaption (DA) allows machine learning methods trained on data sampled from one distribution to be applied to data sampled from another.
TasselNet: Counting maize tassels in the wild via local counts regression network
To our knowledge, this is the first time that a plant-related counting problem is considered using computer vision technologies under unconstrained field-based environment.
