3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation

The ability to perceive and understand 3D scenes is crucial for many applications in computer vision and robotics. Inverse graphics is an appealing approach to 3D scene understanding that aims to infer the 3D scene structure from 2D images. In this paper, we introduce probabilistic modeling to the inverse graphics framework to quantify uncertainty and achieve robustness in 6D pose estimation tasks. Specifically, we propose 3D Neural Embedding Likelihood (3DNEL) as a unified probabilistic model over RGB-D images, and develop efficient inference procedures on 3D scene descriptions. 3DNEL effectively combines learned neural embeddings from RGB with depth information to improve robustness in sim-to-real 6D object pose estimation from RGB-D images. Performance on the YCB-Video dataset is on par with state-of-the-art yet is much more robust in challenging regimes. In contrast to discriminative approaches, 3DNEL's probabilistic generative formulation jointly models multiple objects in a scene, quantifies uncertainty in a principled way, and handles object pose tracking under heavy occlusion. Finally, 3DNEL provides a principled framework for incorporating prior knowledge about the scene and objects, which allows natural extension to additional tasks like camera pose tracking from video.

PDF Abstract ICCV 2023 PDF ICCV 2023 Abstract


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
YCB-Video 3DNEL MSIGP Average Recall 84.85% # 1
YCB-Video MegaPose Average Recall 63.3% # 6
YCB-Video FFB6D Average Recall 75.80% # 4
YCB-Video Coupled Iterative Refinement Average Recall 76.58% # 3
YCB-Video CosyPose Average Recall 71.42% # 5
YCB-Video SurfEMB Average Recall 80.00% # 2


No methods listed for this paper. Add relevant methods here