Search Results for author:
Found 88 papers, 32 papers with code
OpticFusion: Multi-Modal Neural Implicit 3D Reconstruction of Microstructures by Fusing White Light Interferometry and Optical Microscopy
However, conventional WLI cannot capture the natural color of a sample's surface, which is essential for many microscale research applications that require both 3D geometry and color information.
StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generation
Recent advances in large reconstruction and generative models have significantly improved scene reconstruction and novel view generation.
CoSurfGS:Collaborative 3D Surface Gaussian Splatting with Distributed Learning for Large Scene Reconstruction
3D Gaussian Splatting (3DGS) has demonstrated impressive performance in scene reconstruction.
GeoTexDensifier: Geometry-Texture-Aware Densification for High-Quality Photorealistic 3D Gaussian Splatting
3D Gaussian Splatting (3DGS) has recently attracted wide attentions in various areas such as 3D navigation, Virtual Reality (VR) and 3D simulation, due to its photorealistic and efficient rendering performance.
3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark
We benchmark a wide range of open-sourced and proprietary LMMs, uncovering their limitations in various aspects of 3D awareness, such as height, orientation, location, and multi-object reasoning, as well as their degraded performance on images with uncommon camera viewpoints.
DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wild
To tackle these challenges, we propose a dynamic-aware tracking any point (DATAP) method that leverages consistent video depth and point tracking.
A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding
We integrate pose embedding to encapsulate information such as multi-view camera poses, providing implicit geometric constraints for multi-view disparity feature fusion dominated by attention.
XRDSLAM: A Flexible and Modular Framework for Deep Learning based SLAM
In this paper, we propose a flexible SLAM framework, XRDSLAM.
DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion
Then, we propose a novel text-driven panoramic generation framework, termed DiffPano, to achieve scalable, consistent, and diverse panoramic scene generation.
ETO:Efficient Transformer-based Local Feature Matching by Organizing Multiple Homography Hypotheses
On the YFCC100M dataset, our matching accuracy is competitive with LoFTR, a state-of-the-art transformer-based architecture, while the inference speed is boosted to 4 times, even outperforming the CNN-based methods.
LiVisSfM: Accurate and Robust Structure-from-Motion with LiDAR and Visual Cues
This paper presents an accurate and robust Structure-from-Motion (SfM) pipeline named LiVisSfM, which is an SfM-based reconstruction system that fully combines LiDAR and visual cues.
BlinkVision: A Benchmark for Optical Flow, Scene Flow and Point Tracking Estimation using RGB Frames and Events
BlinkVision enables extensive benchmarks on three types of correspondence tasks (optical flow, point tracking, and scene flow estimation) for both image-based and event-based methods, offering new observations, practices, and insights for future research.
StreetSurfGS: Scalable Urban Street Surface Reconstruction with Planar-based Gaussian Splatting
Reconstructing urban street scenes is crucial due to its vital role in applications such as autonomous driving and urban planning.
BlinkTrack: Feature Tracking over 100 FPS via Events and Images
Feature tracking is crucial for, structure from motion (SFM), simultaneous localization and mapping (SLAM), object tracking and various computer vision tasks.
SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality
Visual localization plays an important role in the applications of Augmented Reality (AR), which enable AR devices to obtain their 6-DoF pose in the pre-build map in order to render virtual content in real scenes.
GigaGS: Scaling up Planar-Based 3D Gaussians for Large Scene Surface Reconstruction
To this end, we propose GigaGS, the first work for high-quality surface reconstruction for large-scale scenes using 3DGS.
MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds
Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks such as denoising real-world human mocap data, recovering human motion from partial 3D observations, mitigating jitters for SMPL-based pose estimators, and refining the results of motion in-betweening.
ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction
Neural implicit reconstruction via volume rendering has demonstrated its effectiveness in recovering dense 3D surfaces.
NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction
Signed Distance Function (SDF)-based volume rendering has demonstrated significant capabilities in surface reconstruction.
NIS-SLAM: Neural Implicit Semantic RGB-D SLAM for 3D Consistent Scene Understanding
However, a notable gap exists in the existing approaches when it comes to scene understanding.
Scene Understanding
Simultaneous Localization and Mapping
+1
ImageNet3D: Towards General-Purpose Object-Level 3D Understanding
A vision model with general-purpose object-level 3D understanding should be capable of inferring both 2D (e. g., class name and bounding box) and 3D information (e. g., 3D location and 3D viewpoint) for arbitrary rigid objects in natural images.
Image Captioning
Linear Probing Object-Level 3D Awareness
+2
PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction
Experiments on indoor and outdoor scenes show that our method achieves fast training and rendering while maintaining high-fidelity rendering and geometric reconstruction, outperforming 3DGS-based and NeRF-based methods.
On the analysis of a higher-order Lotka-Volterra model: an application of S-tensors and the polynomial complementarity problem
To address this challenge, we introduce HOIs (Higher-Order Interactions) which are able to capture, for example, the indirect effect of one species on a second one correlating to a third species.
GScream: Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal
This paper tackles the intricate challenge of object removal to update the radiance field using the 3D Gaussian Splatting.
GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image
Recently, we have witnessed the explosive growth of various volumetric representations in modeling animatable head avatars.
CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Field
Recently neural radiance fields (NeRF) have been widely exploited as 3D representations for dense simultaneous localization and mapping (SLAM).
Vox-Fusion++: Voxel-based Neural Implicit Dense Tracking and Mapping with Multi-maps
In this paper, we introduce Vox-Fusion++, a multi-maps-based robust dense tracking and mapping system that seamlessly fuses neural implicit representations with traditional volumetric fusion techniques.
On discrete-time polynomial dynamical systems on hypergraphs
Next, we extend our results to non-homogeneous polynomial systems on non-uniform hypergraphs.
Multi-View Neural 3D Reconstruction of Micro-/Nanostructures with Atomic Force Microscopy
However, conventional AFM scanning struggles to reconstruct complex 3D micro-/nanostructures precisely due to limitations such as incomplete sample topography capturing and tip-sample convolution artifacts.
On Metzler positive systems on hypergraphs
To be more precise, in this paper, we first extend the concept of a Metzler matrix to a Metzler tensor and then describe some useful properties of such tensors.
DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement
Furthermore we also develop an uncertainty estimation module within diffusion to evaluate the reliability of estimated scene flow.
DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Diffusion Model
Furthermore, we also develop an uncertainty estimation module within diffusion to evaluate the reliability of estimated scene flow.
Hard Label Black Box Node Injection Attack on Graph Neural Networks
In this work, we will propose a non-targeted Hard Label Black Box Node Injection Attack on Graph Neural Networks, which to the best of our knowledge, is the first of its kind.
RD-VIO: Robust Visual-Inertial Odometry for Mobile Augmented Reality in Dynamic Environments
In this work, we design a novel visual-inertial odometry (VIO) system called RD-VIO to handle both of these two problems.
Depth Completion with Multiple Balanced Bases and Confidence for Dense Monocular SLAM
In this paper, we propose a novel method that integrates a light-weight depth completion network into a sparse SLAM system using a multi-basis depth representation, so that dense mapping can be performed online even on a mobile phone.
Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor
Specifically, we propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor which drives the optimization by comparing with the raw sensor inputs.
Graph-based Asynchronous Event Processing for Rapid Object Recognition
Different from traditional video cameras, event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes.
Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.
Ranked #1 on
Animal Pose Estimation
on Animal3D
Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing
Recently, Neural Radiance Fields (NeRF) has exhibited significant success in novel view synthesis, surface reconstruction, etc.
Generating Images with 3D Annotations Using Diffusion Models
With explicit 3D geometry control, we can easily change the 3D structures of the objects in the generated images and obtain ground-truth 3D annotations automatically.
DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene Generation
In addition to the scene generation, the final part of DiffInDScene can be used as a post-processing module to refine the 3D reconstruction results from multi-view stereo.
NOVUM: Neural Object Volumes for Robust Object Classification
Discriminative models for object classification typically learn image-based representations that do not capture the compositional and 3D nature of objects.
A bioinspired three-stage model for camouflaged object detection
Furthermore, considering the significance of multi-scale information, we have designed a multi-scale feature enhancement module that enlarges the receptive field while preserving detailed structural cues.
Long-term Visual Localization with Mobile Sensors
Despite the remarkable advances in image matching and pose estimation, image-based localization of a camera in a temporally-varying outdoor environment is still a challenging problem due to huge appearance disparity between query and reference images caused by illumination, seasonal and structural changes.
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field
Despite the great success in 2D editing using user-friendly tools, such as Photoshop, semantic strokes, or even text prompts, similar capabilities in 3D areas are still limited, either relying on 3D modeling skills or allowing editing within only a few categories.
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
Experiments show that BlinkFlow improves the generalization performance of state-of-the-art methods by more than 40\% on average and up to 90\%.
EC-SfM: Efficient Covisibility-based Structure-from-Motion for Both Sequential and Unordered Images
Based on this general image connection, we propose a unified framework to efficiently reconstruct sequential images, unordered images, and the mixture of these two.
Clothed Human Performance Capture With a Double-Layer Neural Radiance Fields
Previous methods capture the performance of full humans with a personalized template or recover the garments from a single frame with static human poses.
DPS-Net: Deep Polarimetric Stereo Depth Estimation
Stereo depth estimation usually struggles to deal with textureless scenes for both traditional and learning-based methods due to the inherent dependence on image correspondence matching.
Deep Active Contours for Real-time 6-DoF Object Tracking
Specifically, given an initial pose, we project the object model to the image plane to obtain the initial contour and use a lightweight network to predict how the contour should move to match the true object boundary, which provides the gradients to optimize the object pose.
Improving Feature-based Visual Localization by Geometry-Aided Matching
We apply GAM to a new hierarchical visual localization pipeline and show that GAM can effectively improve the robustness and accuracy of localization.
Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation
In this work, we present a dense tracking and mapping system named Vox-Fusion, which seamlessly fuses neural implicit representations with traditional volumetric fusion methods.
Hierarchical and Progressive Image Matting
In this paper, we propose an end-to-end Hierarchical and Progressive Attention Matting Network (HAttMatting++), which can better predict the opacity of the foreground from single RGB images without additional input.
Generative Category-Level Shape and Pose Estimation with Semantic Primitives
Then, by using a SIM(3)-invariant shape descriptor, we gracefully decouple the shape and pose of an object, thus supporting latent shape optimization of target objects in arbitrary poses.
Ranked #2 on
6D Pose Estimation using RGBD
on REAL275
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis
Existing inverse rendering combined with neural rendering methods can only perform editable novel view synthesis on object-specific scenes, while we present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method and can extend its application to room-scale scenes.
DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image
Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc.
NeuralMarker: A Framework for Learning General Marker Correspondence
We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker.
Vox-Surf: Voxel-based Implicit Surface Representation
Virtual content creation and interaction play an important role in modern 3D applications such as AR and VR.
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing
Very recently neural implicit rendering techniques have been rapidly evolved and shown great advantages in novel view synthesis and 3D scene reconstruction.
D$^3$FlowSLAM: Self-Supervised Dynamic SLAM with Flow Motion Decomposition and DINO Guidance
We propose a dynamic update module based on this representation and develop a dense SLAM system that excels in dynamic scenarios.
Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation
Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications.
VIP-SLAM: An Efficient Tightly-Coupled RGB-D Visual Inertial Planar SLAM
Even if the plane parameters are involved in the optimization, we effectively simplify the back-end map by using planar structures.
PVO: Panoptic Visual Odometry
We present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information.
OnePose: One-Shot Object Pose Estimation without CAD Models
We propose a new method named OnePose for object pose estimation.
Neural 3D Scene Reconstruction with the Manhattan-world Assumption
Based on the Manhattan-world assumption, planar constraints are employed to regularize the geometry in floor and wall regions predicted by a 2D semantic segmentation network.
Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects
We, as human beings, can understand and picture a familiar scene from arbitrary viewpoints given a single image, whereas this is still a grand challenge for computers.
RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization
The correspondence field estimation and pose refinement are conducted alternatively in each iteration to recover the object poses.
Ranked #1 on
6D Pose Estimation using RGB
on LineMOD
Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation
HybridTracker performs pixel tracker and instance tracker in parallel to obtain the association matrices, which are fused into a matching matrix.
Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling
The correct ego-motion estimation basically relies on the understanding of correspondences between adjacent LiDAR scans.
CGiS-Net: Aggregating Colour, Geometry and Implicit Semantic Features for Indoor Place Recognition
We describe a novel approach to indoor place recognition from RGB point clouds based on aggregating low-level colour and geometry features with high-level implicit semantic features.
Efficient Quantum Feature Extraction for CNN-based Learning
Recent work has begun to explore the potential of parametrized quantum circuits (PQCs) as general function approximators.
SceneSqueezer: Learning To Compress Scene for Camera Relocalization
Standard visual localization methods build a priori 3D model of a scene which is used to establish correspondences against the 2D keypoints in a query image.
Deep-CNN based Robotic Multi-Class Under-Canopy Weed Control in Precision Farming
Smart weeding systems to perform plant-specific operations can contribute to the sustainability of agriculture and the environment.
LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies
In this work, we propose a novel neural implicit representation for the human body, which is fully differentiable and optimizable with disentangled shape and pose latent spaces.
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
In this paper, we present a novel neural scene rendering system, which learns an object-compositional neural radiance field and produces realistic rendering with editing capability for a clustered and real-world scene.
VS-Net: Voting with Segmentation for Visual Localization
To address this problem, we propose a novel visual localization framework that establishes 2D-to-3D correspondences between the query image and the 3D map with a series of learnable scene-specific landmarks.
LIFE: Lighting Invariant Flow Estimation
However, local image contents are inevitably ambiguous and error-prone during the cross-image feature matching process, which hinders downstream tasks.
Detecting quantum entanglement with unsupervised learning
Quantum properties, such as entanglement and coherence, are indispensable resources in various quantum information processing tasks.
You Don't Only Look Once: Constructing Spatial-Temporal Memory for Integrated 3D Object Detection and Tracking
In this work, we propose a novel system for integrated 3D object detection and tracking, which uses a dynamic object occupancy map and previous object states as spatial-temporal memory to assist object detection in future frames.
Super interference fringes of two-photon photoluminescence in individual Au nanoparticles: the critical role of the intermediate state
Here, we presented two-photon photoluminescence (TPPL) measurements on individual Au nanobipyramids (AuNP) to reveal their ultrafast dynamics by two-pulse excitation on a global time scale ranging from sub-femtosecond to tens of picoseconds.
Optics Mesoscale and Nanoscale Physics Quantum Physics
SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks
To suit our network to self-supervised learning, we design several novel loss functions that utilize the inherent properties of LiDAR point clouds.
Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone
The proposed mesh generation module incrementally fuses each estimated keyframe depth map to an online dense surface mesh, which is useful for achieving realistic AR effects such as occlusions and collisions.
Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints
Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises.
Quantum tensor singular value decomposition with applications to recommendation systems
In this paper, we present a quantum singular value decomposition algorithm for third-order tensors inspired by the classical algorithm of tensor singular value decomposition (t-svd) and then extend it to order-$p$ tensors.
ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM
However, jointly using visual and inertial measurements to optimize SLAM objective functions is a problem of high computational complexity.
Robust Keyframe-based Dense SLAM with an RGB-D Camera
In this paper, we present RKD-SLAM, a robust keyframe-based dense SLAM approach for an RGB-D camera that can robustly handle fast motion and dense loop closure, and run without time limitation in a moderate size scene.
ENFT: Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion
Our framework consists of steps of solving the feature `dropout' problem when indistinctive structures, noise or large image distortion exists, and of rapidly recognizing and joining common features located in different subsequences.
