Search Results for author:
Found 44 papers, 21 papers with code
Boost 3D Reconstruction using Diffusion-based Monocular Camera Calibration
Monocular camera calibration is essential for many 3D vision tasks.
Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving
In contrast, Large Vision-Language Models (LVLMs) excel in scene understanding and reasoning.
Depth Any Video with Scalable Synthetic Data
Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results.
DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model
The ability of this world model to capture the evolution of the environment is crucial for planning in autonomous driving.
HE-Drive: Human-Like End-to-End Driving with Vision Language Models
In this paper, we propose HE-Drive: the first human-like-centric end-to-end autonomous driving system to generate trajectories that are both temporally consistent and comfortable.
OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexity
In this paper, we introduce OccRWKV, an efficient semantic occupancy network inspired by Receptance Weighted Key Value (RWKV).
Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Prediction
In this paper, we provide a systemic analysis of the diffusion formulation for the dense prediction, focusing on both quality and efficiency.
DC-Gaussian: Improving 3D Gaussian Splatting for Reflective Dash Cam Videos
Dash cam videos often suffer from severe obstructions such as reflections and occlusions on the windshields, which significantly impede the application of neural rendering techniques.
Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
For metric depth estimation, we show that the key to a zero-shot single-view model lies in resolving the metric ambiguity from various camera models and large-scale data training.
Ranked #1 on
Surface Normals Estimation
on NYU Depth v2
(using extra training data)
LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment
Language-guided scene-aware human motion generation has great significance for entertainment and robotics.
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image
We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes, e. g., depth and normals, from single images.
Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving
In this work, we propose a new robustness benchmark to evaluate the depth estimation system under various noisy pose settings.
Ranked #1 on
Monocular Depth Estimation
on DDAD
GaussianPro: 3D Gaussian Splatting with Progressive Propagation
The advent of 3D Gaussian Splatting (3DGS) has recently brought about a revolution in the field of neural rendering, facilitating high-quality renderings at real-time speed.
SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets
We suggest building virtual pinhole cameras to resolve the distortion problem of fisheye cameras and unify the processing for the two types of 360$^\circ$ cameras.
GIM: Learning Generalizable Image Matcher From Internet Videos
Given an architecture, GIM first trains it on standard domain-specific datasets and then combines it with complementary matching methods to create dense labels on nearby frames of novel videos.
PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion
However, the generation quality and generalization ability of 3D diffusion models is hindered by the scarcity of high-quality and large-scale 3D datasets.
UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving
Multi-camera setups find widespread use across various applications, such as autonomous driving, as they greatly expand sensing capabilities.
HumanRecon: Neural Reconstruction of Dynamic Human Using Geometric Cues and Physical Priors
These priors ensure the color rendered along rays to be robust to view direction and reduce the inherent ambiguities of density estimated along rays.
Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering
In this paper, we propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations.
FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models
3D scene reconstruction is a long-standing vision task.
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image
State-of-the-art (SOTA) monocular metric depth estimation methods can only handle a single camera model and are unable to perform mixed-data training due to the metric ambiguity.
Ranked #25 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes
To let the geometric perception learned from multi-view cues in static areas propagate to the monocular representation in dynamic areas and let monocular cues enhance the representation of multi-view cost volume, we propose a cross-cue fusion (CCF) module, which includes the cross-cue attention (CCA) to encode the spatially non-local relative intra-relations from each source to enhance the representation of the other.
The Second Monocular Depth Estimation Challenge
This paper discusses the results for the second edition of the Monocular Depth Estimation Challenge (MDEC).
Distributed Graph Embedding with Information-Oriented Random Walks
Graph embedding maps graph nodes to low-dimensional vectors, and is widely adopted in machine learning tasks.
SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes
Self-supervised monocular depth estimation has shown impressive results in static scenes.
Indoor Monocular Depth Estimation
Monocular Depth Estimation
+1
Hierarchical Normalization for Robust Monocular Depth Estimation
In this paper, we address monocular depth estimation with deep neural networks.
Ranked #7 on
Monocular Depth Estimation
on ETH3D
Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image
To do so, we propose a two-stage framework that first predicts depth up to an unknown scale and shift from a single monocular image, and then exploits 3D point cloud data to predict the depth shift and the camera's focal length that allow us to recover 3D scene shapes.
Towards Domain-agnostic Depth Completion
Our method leverages a data-driven prior in the form of a single image depth prediction network trained on large-scale datasets, the output of which is used as an input to our model.
Controllable Shadow Generation Using Pixel Height Maps
It can be used to calculate hard shadows in a 2D image based on the projective geometry, providing precise control of the shadows' direction and shape.
Exploiting Correspondences with All-pairs Correlations for Multi-view Depth Estimation
However, multi-view depth estimation is fundamentally a correspondence-based optimization problem, but previous learning-based methods mainly rely on predefined depth hypotheses to build correspondence as the cost volume and implicitly regularize it to fit depth prediction, deviating from the essence of iterative optimization based on stereo correspondence.
PointInst3D: Segmenting 3D Instances by Points
The current state-of-the-art methods in 3D instance segmentation typically involve a clustering step, despite the tendency towards heuristics, greedy algorithms, and a lack of robustness to the changes in data statistics.
Improving Monocular Visual Odometry Using Learned Depth
The core of our framework is a monocular depth estimation module with a strong generalization capability for diverse scenes.
Retrieval Augmented Classification for Long-Tail Visual Recognition
We introduce Retrieval Augmented Classification (RAC), a generic approach to augmenting standard image classification pipelines with an explicit retrieval module.
Ranked #6 on
Long-tail Learning
on iNaturalist 2018
Scaling up Multi-domain Semantic Segmentation with Sentence Embeddings
The resulting merged semantic segmentation dataset of over 2 Million images enables training a model that achieves performance equal to that of state-of-the-art supervised methods on 7 benchmark datasets, despite not using any images therefrom.
Ranked #1 on
Semantic Segmentation
on WildDash
Towards 3D Scene Reconstruction from Locally Scale-Aligned Monocular Video Depth
However, in some video-based scenarios such as video depth estimation and 3D scene reconstruction from a video, the unknown scale and shift residing in per-frame prediction may cause the depth inconsistency.
Pseudo-LiDAR Based Road Detection
Specifically, we exploit pseudo-LiDAR using depth estimation, and propose a feature fusion network where RGB and learned depth information are fused for improved road detection.
Generic Perceptual Loss for Modeling Structured Output Dependencies
We hope that this simple, extended perceptual loss may serve as a generic structured-output loss that is applicable to most structured output learning tasks.
Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction
In this work, we show the importance of the high-order 3D geometric constraints for depth prediction.
Learning to Recover 3D Scene Shape from a Single Image
Despite significant progress in monocular depth estimation in the wild, recent state-of-the-art methods cannot be used to recover accurate 3D scene shape due to an unknown depth shift induced by shift-invariant reconstruction losses used in mixed-data depth prediction training, and possible unknown camera focal length.
Ranked #1 on
Indoor Monocular Depth Estimation
on DIODE
(using extra training data)
Hierarchical Text Interaction for Rating Prediction
At review level, we mutually propagate textual features between the user and item, and capture the informative reviews.
DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data
Compared with previous learning objectives, i. e., learning metric depth or relative depth, we propose to learn the affine-invariant depth using our diverse dataset to ensure both generalization and high-quality geometric shapes of scenes.
Task-Aware Monocular Depth Estimation for 3D Object Detection
In this paper, we first analyse the data distributions and interaction of foreground and background, then propose the foreground-background separated monocular depth estimation (ForeSeE) method, to estimate the foreground depth and background depth using separate optimization objectives and depth decoders.
Auxiliary Learning for Deep Multi-task Learning
The most current methods can be categorized as either: (i) hard parameter sharing where a subset of the parameters is shared among tasks while other parameters are task-specific; or (ii) soft parameter sharing where all parameters are task-specific but they are jointly regularized.
Enforcing geometric constraints of virtual normal for depth prediction
Monocular depth prediction plays a crucial role in understanding 3D scene geometry.
Ranked #10 on
Depth Estimation
on NYU-Depth V2
