Search Results for author:
Found 47 papers, 18 papers with code
ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Grasping
A key insight of our method is that occlusion reasoning and modeling the spatial relationships between objects is beneficial for both accurate reconstruction and grasping.
Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion
Our method uses raymap conditioning to both augment visual features with spatial information from different viewpoints, as well as to guide the generation of images and depth maps from novel views.
PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
While Vision-Language Models (VLMs) have shown great promise in reasoning and task planning for embodied agents, their ability to comprehend physical phenomena remains extremely limited.
Learning from Massive Human Videos for Universal Humanoid Pose Control
Scalable learning of humanoid robots is crucial for their deployment in real-world applications.
$SE(3)$ Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation
Incorporating inductive bias by embedding geometric entities (such as rays) as input has proven successful in multi-view learning.
GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion
3D reconstruction from a single image is a long-standing problem in computer vision.
Ranked #8 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
View-Invariant Policy Learning via Zero-Shot Novel View Synthesis
For practical application to diverse robotic data, these models must operate zero-shot, performing view synthesis on unseen tasks and environments.
Incorporating dense metric depth into neural 3D representations for view synthesis and relighting
Synthesizing accurate geometry and photo-realistic appearance of small scenes is an active area of research with compelling use cases in gaming, virtual reality, robotic-manipulation, autonomous driving, convenient product capture, and consumer-level photography.
Learning Temporally Consistent Video Depth from Video Diffusion Priors
Moreover, we design an effective training strategy to provide context within a clip.
Self-Supervised Geometry-Guided Initialization for Robust Monocular Visual Odometry
Monocular visual odometry is a key technology in a wide variety of autonomous systems.
Transcrib3D: 3D Referring Expression Resolution through Large Language Models
We introduce Transcrib3D, an approach that brings together 3D detection methods and the emergent reasoning capabilities of large language models (LLMs).
NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields
Given the capabilities of neural fields in densely representing a 3D scene from 2D images, we ask the question: Can we scale their self-supervised pretraining, specifically using masked autoencoders, to generate effective 3D representations from posed RGB images.
Towards Realistic Scene Generation with LiDAR Diffusion Models
In this paper, we propose LiDAR Diffusion Models (LiDMs) to generate LiDAR-realistic scenes from a latent space tailored to capture the realism of LiDAR scenes by incorporating geometric priors into the learning pipeline.
Zero-Shot Multi-Object Scene Completion
We present a 3D scene completion method that recovers the complete geometry of multiple unseen objects in complex scenes from a single RGB-D image.
NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes
NeO 360's representation allows us to learn from a large collection of unbounded 3D scenes while offering generalizability to new views and novel scenes from as few as a single image during inference.
Ranked #1 on
Generalizable Novel View Synthesis
on NERDS 360
Robust Self-Supervised Extrinsic Self-Calibration
Autonomous vehicles and robots need to operate over a wide variety of scenarios in order to complete tasks efficiently and safely.
Towards Zero-Shot Scale-Aware Monocular Depth Estimation
Monocular depth estimation is scale-ambiguous, and thus requires scale supervision to produce metric predictions.
NeRFuser: Large-Scale Scene Representation by NeRF Fusion
A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency: large scenes can be efficiently stored and shared as small neural nets instead of collections of images.
DeLiRa: Self-Supervised Depth, Light, and Radiance Fields
In this work, we propose to use the multi-view photometric objective from the self-supervised depth estimation literature as a geometric regularizer for volumetric rendering, significantly improving novel view synthesis without requiring additional information.
Viewpoint Equivariance for Multi-View 3D Object Detection
We design view-conditioned queries at the output level, which enables the generation of multiple virtual frames during training to learn viewpoint equivariance by enforcing multi-view consistency.
Photo-realistic Neural Domain Randomization
In this paper, we show that the recent progress in neural rendering enables a new unified approach we call Photo-realistic Neural Domain Randomization (PNDR).
Depth Is All You Need for Monocular 3D Detection
Our methods leverage commonly available LiDAR or RGB videos during training time to fine-tune the depth representation, which leads to improved 3D detectors.
Depth Field Networks for Generalizable Multi-view Scene Representation
Modern 3D computer vision leverages learning to boost geometric reasoning, mapping image data to classical structures such as cost volumes or epipolar constraints to improve matching.
SpOT: Spatiotemporal Modeling for 3D Object Tracking
In this work, we develop a holistic representation of traffic scenes that leverages both spatial and temporal information of the actors in the scene.
Multi-Frame Self-Supervised Depth with Transformers
Experiments on the KITTI and DDAD datasets show that our DepthFormer architecture establishes a new state of the art in self-supervised monocular depth estimation, and is even competitive with highly specialized supervised single-frame architectures.
Learning Optical Flow, Depth, and Scene Flow without Real-World Labels
However, the simultaneous self-supervised learning of depth and scene flow is ill-posed, as there are infinitely many combinations that result in the same 3D point.
Self-Supervised Camera Self-Calibration from Video
Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams.
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries
This top-down approach outperforms its bottom-up counterpart in which object bounding box prediction follows per-pixel depth estimation, since it does not suffer from the compounding error introduced by a depth prediction model.
Ranked #7 on
Robust Camera Only 3D Object Detection
on nuScenes-C
Is Pseudo-Lidar needed for Monocular 3D Object detection?
Recent progress in 3D object detection from single images leverages monocular depth estimation as a way to produce 3D pointclouds, turning cameras into pseudo-lidar sensors.
Ranked #1 on
Monocular 3D Object Detection
on KITTI Pedestrian Moderate
(using extra training data)
MarioNette: Self-Supervised Sprite Learning
Artists and video game designers often construct 2D animations using libraries of sprites -- textured patches of objects and characters.
Full Surround Monodepth from Multiple Cameras
In this work, we extend monocular self-supervised depth and ego-motion estimation to large-baseline multi-camera rigs.
Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion
Estimating scene geometry from data obtained with cost-effective sensors is key for robots and self-driving cars.
Geometric Unsupervised Domain Adaptation for Semantic Segmentation
Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation.
Monocular Depth Estimation for Soft Visuotactile Sensors
Fluid-filled soft visuotactile sensors such as the Soft-bubbles alleviate key challenges for robust manipulation, as they enable reliable grasps along with the ability to obtain high-resolution sensory feedback on contact geometry and forces.
Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion
Self-supervised learning has emerged as a powerful tool for depth and ego-motion estimation, leading to state-of-the-art results on benchmark datasets.
Semantically-Guided Representation Learning for Self-Supervised Monocular Depth
Instead of using semantic labels and proxy losses in a multi-task approach, we propose a new architecture leveraging fixed pretrained semantic segmentation networks to guide self-supervised representation learning via pixel-adaptive convolutions.
Neural Outlier Rejection for Self-Supervised Keypoint Learning
By making the sampling of inlier-outlier sets from point-pair correspondences fully differentiable within the keypoint learning framework, we show that are able to simultaneously self-supervise keypoint description and improve keypoint matching.
Self-Supervised 3D Keypoint Learning for Ego-motion Estimation
Detecting and matching robust viewpoint-invariant keypoints is critical for visual SLAM and Structure-from-Motion.
Dynamic Hilbert Maps: Real-Time Occupancy Predictions in Changing Environment
This paper addresses the problem of learning instantaneous occupancy levels of dynamic environments and predicting future occupancy levels.
Real-Time Panoptic Segmentation from Dense Detections
Panoptic segmentation is a complex full scene parsing task requiring simultaneous instance and semantic segmentation at high resolution.
Two Stream Networks for Self-Supervised Ego-Motion Estimation
Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues.
Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances
Dense depth estimation from a single image is a key problem in computer vision, with exciting applications in a multitude of robotic tasks.
3D Packing for Self-Supervised Monocular Depth Estimation
Although cameras are ubiquitous, robotic platforms typically rely on active sensors like LiDAR for direct 3D perception.
Sparse-to-Continuous: Enhancing Monocular Depth Estimation using Occupancy Maps
This paper addresses the problem of single image depth estimation (SIDE), focusing on improving the quality of deep neural network predictions.
Learning to Race through Coordinate Descent Bayesian Optimisation
On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system.
Continuous Convolutional Neural Networks for Image Classification
This paper introduces the concept of continuous convolution to neural networks and deep learning applications in general.
Bayesian Optimisation for Safe Navigation under Localisation Uncertainty
In outdoor environments, mobile robots are required to navigate through terrain with varying characteristics, some of which might significantly affect the integrity of the platform.
