Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces

no code implementations • 29 Mar 2024 • Mauro Comi, Alessio Tonioni, Max Yang, Jonathan Tremblay, Valts Blukis, Yijiong Lin, Nathan F. Lepora, Laurence Aitchison

Touch and vision go hand in hand, mutually enhancing our ability to understand the world.

Novel View Synthesis Surface Reconstruction

TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing

no code implementations • 21 Nov 2023 • Mauro Comi, Yijiong Lin, Alex Church, Alessio Tonioni, Laurence Aitchison, Nathan F. Lepora

To address these challenges, we propose TouchSDF, a Deep Learning approach for tactile 3D shape reconstruction that leverages the rich information provided by a vision-based tactile sensor and the expressivity of the implicit neural representation DeepSDF.

3D Shape Reconstruction

Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control

no code implementations • 26 Jul 2023 • Yijiong Lin, Mauro Comi, Alex Church, Dandan Zhang, Nathan F. Lepora

To improve the robustness of tactile robot control in unstructured environments, we propose and study a new concept: \textit{tactile saliency} for robot touch, inspired by the human touch attention mechanism from neuroscience and the visual saliency prediction problem from computer vision.

Pose Estimation Saliency Prediction