Search Results for author: Luca Carlone

Found 52 papers, 34 papers with code

Shonan Rotation Averaging: Global Optimality by Surfing SO(p)(n)

no code implementations ECCV 2020 Frank Dellaert, David M. Rosen, Jing Wu, Robert Mahony, Luca Carlone

Shonan Rotation Averaging is a fast, simple, and elegant rotation averaging algorithm that is guaranteed to recover globally optimal solutions under mild assumptions on the measurement noise.

Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds

no code implementations16 Feb 2024 David Jin, Sushrut Karmalkar, Harry Zhang, Luca Carlone

We investigate a variation of the 3D registration problem, named multi-model 3D registration.

Vision-Based Terrain Relative Navigation on High-Altitude Balloon and Sub-Orbital Rocket

no code implementations16 Feb 2023 Dominic Maggio, Courtney Mario, Brett Streetman, Ted Steiner, Luca Carlone

Additionally, we evaluate performance on data collected by two cameras inside the capsule of Blue Origin's New Shepard rocket on payload flight NS-23, traveling at speeds up to 880 km/hr, and demonstrate less than 55 meters of average position error.


A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training

no code implementations12 Feb 2023 Jingnan Shi, Rajat Talak, Dominic Maggio, Luca Carlone

Real-world robotics applications demand object pose estimation methods that work reliably across a variety of scenarios.

Keypoint Detection

NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields

1 code implementation24 Oct 2022 Antoni Rosinol, John J. Leonard, Luca Carlone

We propose a novel geometric and photometric 3D mapping pipeline for accurate and real-time scene reconstruction from monocular images.

Probabilistic Volumetric Fusion for Dense Monocular SLAM

1 code implementation3 Oct 2022 Antoni Rosinol, John J. Leonard, Luca Carlone

We present a novel method to reconstruct 3D scenes from images by leveraging deep dense monocular SLAM and fast uncertainty propagation.

Estimation Contracts for Outlier-Robust Geometric Perception

1 code implementation22 Aug 2022 Luca Carlone

In particular, we adapt and extend recent results on robust linear regression (applicable to the low-outlier regime with << 50% outliers) and list-decodable regression (applicable to the high-outlier regime with >> 50% outliers) to the setup commonly found in robotics and vision, where (i) variables (e. g., rotations, poses) belong to a non-convex domain, (ii) measurements are vector-valued, and (iii) the number of outliers is not known a priori.


Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints

1 code implementation24 Jun 2022 Jingnan Shi, Heng Yang, Luca Carlone

We consider an active shape model, where -- for an object category -- we are given a library of potential CAD models describing objects in that category, and we adopt a standard formulation where pose and shape are estimated from 2D or 3D keypoints via non-convex optimization.

Vehicle Pose Estimation

Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Training

2 code implementations22 Jun 2022 Rajat Talak, Lisa Peng, Luca Carlone

Our third contribution is a novel self-supervised training approach that uses our certificate of observable correctness to provide the supervisory signal to C-3PO during training.

Pose Estimation

Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems

no code implementations25 Sep 2021 Amanda Prorok, Matthew Malencia, Luca Carlone, Gaurav S. Sukhatme, Brian M. Sadler, Vijay Kumar

In this survey article, we analyze how resilience is achieved in networks of agents and multi-robot systems that are able to overcome adversity by leveraging system-wide complementarity, diversity, and redundancy - often involving a reconfiguration of robotic capabilities to provide some key ability that was not present in the system a priori.

Certifiably Optimal Outlier-Robust Geometric Perception: Semidefinite Relaxations and Scalable Global Optimization

2 code implementations7 Sep 2021 Heng Yang, Luca Carlone

Our third contribution is to solve the SDP relaxations at an unprecedented scale and accuracy by presenting STRIDE, a solver that blends global descent on the convex SDP with fast local search on the nonconvex POP.

Pose Estimation

Smooth Mesh Estimation from Depth Data using Non-Smooth Convex Optimization

no code implementations6 Aug 2021 Antoni Rosinol, Luca Carlone

In this paper, we leapfrog these intermediate representations and build a 3D mesh directly from a depth map and the sparse landmarks triangulated with visual odometry.

Visual Odometry

Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems

1 code implementation28 Jun 2021 Yulun Tian, Yun Chang, Fernando Herrera Arias, Carlos Nieto-Granda, Jonathan P. How, Luca Carlone

This paper presents Kimera-Multi, the first multi-robot system that (i) is robust and capable of identifying and rejecting incorrect inter and intra-robot loop closures resulting from perceptual aliasing, (ii) is fully distributed and only relies on local (peer-to-peer) communication to achieve distributed localization and mapping, and (iii) builds a globally consistent metric-semantic 3D mesh model of the environment in real-time, where faces of the mesh are annotated with semantic labels.

3D Reconstruction Benchmarking +1

An Inexact Projected Gradient Method with Rounding and Lifting by Nonlinear Programming for Solving Rank-One Semidefinite Relaxation of Polynomial Optimization

1 code implementation28 May 2021 Heng Yang, Ling Liang, Luca Carlone, Kim-Chuan Toh

In particular, we first design a globally convergent inexact projected gradient method (iPGM) for solving the SDP that serves as the backbone of our framework.

Neural Trees for Learning on Graphs

no code implementations NeurIPS 2021 Rajat Talak, Siyi Hu, Lisa Peng, Luca Carlone

We also prove that the number of parameters needed to achieve an $\epsilon$-approximation of the distribution function is exponential in the treewidth of the input graph, but linear in its size.

Node Classification

Optimal Pose and Shape Estimation for Category-level 3D Object Perception

1 code implementation16 Apr 2021 Jingnan Shi, Heng Yang, Luca Carlone

Our first contribution is to provide the first certifiably optimal solver for pose and shape estimation.

Object Translation +1

NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge

no code implementations21 Mar 2021 Ali Agha, Kyohei Otsu, Benjamin Morrell, David D. Fan, Rohan Thakker, Angel Santamaria-Navarro, Sung-Kyun Kim, Amanda Bouman, Xianmei Lei, Jeffrey Edlund, Muhammad Fadhil Ginting, Kamak Ebadi, Matthew Anderson, Torkom Pailevanian, Edward Terry, Michael Wolf, Andrea Tagliabue, Tiago Stegun Vaquero, Matteo Palieri, Scott Tepsuporn, Yun Chang, Arash Kalantari, Fernando Chavez, Brett Lopez, Nobuhiro Funabiki, Gregory Miles, Thomas Touma, Alessandro Buscicchio, Jesus Tordesillas, Nikhilesh Alatur, Jeremy Nash, William Walsh, Sunggoo Jung, Hanseob Lee, Christoforos Kanellakis, John Mayo, Scott Harper, Marcel Kaufmann, Anushri Dixit, Gustavo Correa, Carlyn Lee, Jay Gao, Gene Merewether, Jairo Maldonado-Contreras, Gautam Salhotra, Maira Saboia Da Silva, Benjamin Ramtoula, Yuki Kubo, Seyed Fakoorian, Alexander Hatteland, Taeyeon Kim, Tara Bartlett, Alex Stephens, Leon Kim, Chuck Bergh, Eric Heiden, Thomas Lew, Abhishek Cauligi, Tristan Heywood, Andrew Kramer, Henry A. Leopold, Chris Choi, Shreyansh Daftry, Olivier Toupet, Inhwan Wee, Abhishek Thakur, Micah Feras, Giovanni Beltrame, George Nikolakopoulos, David Shim, Luca Carlone, Joel Burdick

This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge.

Decision Making Motion Planning

Dynamic Grasping with a "Soft" Drone: From Theory to Practice

no code implementations11 Mar 2021 Joshua Fishman, Samuel Ubellacker, Nathan Hughes, Luca Carlone

This paper presents the first prototype of a soft drone -- a quadrotor where traditional (i. e., rigid) landing gears are replaced with a soft tendon-actuated gripper to enable aggressive grasping.


Self-supervised Geometric Perception

2 code implementations CVPR 2021 Heng Yang, Wei Dong, Luca Carlone, Vladlen Koltun

We present self-supervised geometric perception (SGP), the first general framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels (e. g., camera poses, rigid transformations).

Point Cloud Registration Pose Estimation

Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs

2 code implementations18 Jan 2021 Antoni Rosinol, Andrew Violette, Marcus Abate, Nathan Hughes, Yun Chang, Jingnan Shi, Arjun Gupta, Luca Carlone

This mental model captures geometric and semantic aspects of the scene, describes the environment at multiple levels of abstractions (e. g., objects, rooms, buildings), includes static and dynamic entities and their relations (e. g., a person is in a room at a given time).

3D Reconstruction Object Localization +2

Monitoring and Diagnosability of Perception Systems

no code implementations11 Nov 2020 Pasquale Antonante, David I. Spivak, Luca Carlone

The resulting temporal diagnostic graphs (i) provide a framework to reason over the consistency of perception outputs -- across modules and over time -- thus enabling fault detection, (ii) allow us to establish formal guarantees on the maximum number of faults that can be uniquely identified in a given perception system, and (iii) enable the design of efficient algorithms for fault identification.

Fault Detection

Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mapping

no code implementations8 Nov 2020 Yun Chang, Yulun Tian, Jonathan P. How, Luca Carlone

Our system, dubbed Kimera-Multi, is implemented by a team of robots equipped with visual-inertial sensors, and builds a 3D mesh model of the environment in real-time, where each face of the mesh is annotated with a semantic label (e. g., building, road, objects).

Simultaneous Localization and Mapping

ROBIN: a Graph-Theoretic Approach to Reject Outliers in Robust Estimation using Invariants

1 code implementation7 Nov 2020 Jingnan Shi, Heng Yang, Luca Carlone

We also show that in practice the maximum k-core of the compatibility graph provides an approximation of the maximum clique, while being faster to compute in large problems.

Primal-Dual Mesh Convolutional Neural Networks

1 code implementation NeurIPS 2020 Francesco Milano, Antonio Loquercio, Antoni Rosinol, Davide Scaramuzza, Luca Carlone

Recent works in geometric deep learning have introduced neural networks that allow performing inference tasks on three-dimensional geometric data by defining convolution, and sometimes pooling, operations on triangle meshes.


Shonan Rotation Averaging: Global Optimality by Surfing $SO(p)^n$

1 code implementation6 Aug 2020 Frank Dellaert, David M. Rosen, Jing Wu, Robert Mahony, Luca Carlone

Shonan Rotation Averaging is a fast, simple, and elegant rotation averaging algorithm that is guaranteed to recover globally optimal solutions under mild assumptions on the measurement noise.

Outlier-Robust Estimation: Hardness, Minimally Tuned Algorithms, and Applications

no code implementations29 Jul 2020 Pasquale Antonante, Vasileios Tzoumas, Heng Yang, Luca Carlone

We extend ADAPT and GNC to the case where the user does not have prior knowledge of the inlier-noise statistics (or the statistics may vary over time) and is unable to guess a reasonable threshold to separate inliers from outliers (as the one commonly used in RANSAC).

object-detection Object Detection

One Ring to Rule Them All: Certifiably Robust Geometric Perception with Outliers

1 code implementation NeurIPS 2020 Heng Yang, Luca Carlone

We propose the first general and practical framework to design certifiable algorithms for robust geometric perception in the presence of a large amount of outliers.

Pose Estimation

Monitoring and Diagnosability of Perception Systems

no code implementations24 May 2020 Pasquale Antonante, David I. Spivak, Luca Carlone

Towards this goal, we draw connections with the literature on self-diagnosability for multiprocessor systems, and generalize it to (i) account for modules with heterogeneous outputs, and (ii) add a temporal dimension to the problem, which is crucial to model realistic perception systems where modules interact over time.

Fault Detection Self-Driving Cars

Online Monitoring for Neural Network Based Monocular Pedestrian Pose Estimation

no code implementations11 May 2020 Arjun Gupta, Luca Carlone

We investigate the problem of online output monitoring for neural networks that estimate 3D human shapes and poses from images.

Pose Estimation Self-Driving Cars

3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans

3 code implementations15 Feb 2020 Antoni Rosinol, Arjun Gupta, Marcus Abate, Jingnan Shi, Luca Carlone

Our second contribution is to provide the first fully automatic Spatial PerceptIon eNgine(SPIN) to build a DSG from visual-inertial data.

3D Reconstruction Human Detection +3

TEASER: Fast and Certifiable Point Cloud Registration

6 code implementations21 Jan 2020 Heng Yang, Jingnan Shi, Luca Carlone

We propose the first fast and certifiable algorithm for the registration of two sets of 3D points in the presence of large amounts of outlier correspondences.

object-detection Object Detection +2

Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping

12 code implementations6 Oct 2019 Antoni Rosinol, Marcus Abate, Yun Chang, Luca Carlone

We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM).

3D Reconstruction Computational Efficiency +1

DOOR-SLAM: Distributed, Online, and Outlier Resilient SLAM for Robotic Teams

1 code implementation26 Sep 2019 Pierre-Yves Lajoie, Benjamin Ramtoula, Yun Chang, Luca Carlone, Giovanni Beltrame

This paper introduces DOOR-SLAM, a fully distributed SLAM system with an outlier rejection mechanism that can work with less conservative parameters.


Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection

4 code implementations18 Sep 2019 Heng Yang, Pasquale Antonante, Vasileios Tzoumas, Luca Carlone

In this paper, we enable the simultaneous use of non-minimal solvers and robust estimation by providing a general-purpose approach for robust global estimation, which can be applied to any problem where a non-minimal solver is available for the outlier-free case.

Pose Estimation

A Quaternion-based Certifiably Optimal Solution to the Wahba Problem with Outliers

4 code implementations ICCV 2019 Heng Yang, Luca Carlone

Our first contribution is to formulate the Wahba problem using a Truncated Least Squares (TLS) cost that is insensitive to a large fraction of spurious correspondences.

Outlier-Robust Spatial Perception: Hardness, General-Purpose Algorithms, and Guarantees

1 code implementation27 Mar 2019 Vasileios Tzoumas, Pasquale Antonante, Luca Carlone

First, we show that even a simple linear instance of outlier rejection is inapproximable: in the worst-case one cannot design a quasi-polynomial time algorithm that computes an approximate solution efficiently.

Pose Estimation

A Polynomial-time Solution for Robust Registration with Extreme Outlier Rates

2 code implementations20 Mar 2019 Heng Yang, Luca Carlone

We propose a robust approach for the registration of two sets of 3D points in the presence of a large amount of outliers.


Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities

5 code implementations4 Mar 2019 Antoni Rosinol, Torsten Sattler, Marc Pollefeys, Luca Carlone

We propose instead to tightly couple mesh regularization and state estimation by detecting and enforcing structural regularities in a novel factor-graph formulation.

3D Reconstruction Simultaneous Localization and Mapping

Accelerated Inference in Markov Random Fields via Smooth Riemannian Optimization

1 code implementation27 Oct 2018 Siyi Hu, Luca Carlone

We show that this approach, named Fast Unconstrained SEmidefinite Solver (FUSES), can solve large problems in milliseconds.

Image Segmentation Riemannian optimization +1

Modeling Perceptual Aliasing in SLAM via Discrete-Continuous Graphical Models

1 code implementation27 Oct 2018 Pierre-Yves Lajoie, Siyi Hu, Giovanni Beltrame, Luca Carlone

Perceptual aliasing is one of the main causes of failure for Simultaneous Localization and Mapping (SLAM) systems operating in the wild.

Robotics 65K05, 62F10, 68T40, 68W40, 68W25, I.2.9; G.1.6

Convex Relaxations for Pose Graph Optimization with Outliers

no code implementations7 Jan 2018 Luca Carlone, Giuseppe C. Calafiore

Pose Graph Optimization involves the estimation of a set of poses from pairwise measurements and provides a formalization for many problems arising in mobile robotics and geometric computer vision.

Sparse Depth Sensing for Resource-Constrained Robots

1 code implementation4 Mar 2017 Fangchang Ma, Luca Carlone, Ulas Ayaz, Sertac Karaman

We address the following question: is it possible to reconstruct the geometry of an unknown environment using sparse and incomplete depth measurements?

Compressive Sensing Depth Estimation +1

Distributed Mapping with Privacy and Communication Constraints: Lightweight Algorithms and Object-based Models

1 code implementation11 Feb 2017 Siddharth Choudhary, Luca Carlone, Carlos Nieto, John Rogers, Henrik I. Christensen, Frank Dellaert

Our field tests show that the combined use of our distributed algorithms and object-based models reduces the communication requirements by several orders of magnitude and enables distributed mapping with large teams of robots in real-world problems.

Sensor Fusion

Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age

2 code implementations19 Jun 2016 Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, Jose Neira, Ian Reid, John J. Leonard

Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it.


On-Manifold Preintegration for Real-Time Visual-Inertial Odometry

2 code implementations8 Dec 2015 Christian Forster, Luca Carlone, Frank Dellaert, Davide Scaramuzza

However, real-time optimization quickly becomes infeasible as the trajectory grows over time, this problem is further emphasized by the fact that inertial measurements come at high rate, hence leading to fast growth of the number of variables in the optimization.


Pose Graph Optimization in the Complex Domain: Lagrangian Duality, Conditions For Zero Duality Gap, and Optimal Solutions

1 code implementation13 May 2015 Giuseppe Calafiore, Luca Carlone, Frank Dellaert

Our analysis shows that the duality gap is connected to the number of eigenvalues of the penalized pose graph matrix, which arises from the solution of the dual.

Robotics 68W01, 68W40, 68W25, 49K30 I.2.9; G.1.6

Cannot find the paper you are looking for? You can Submit a new open access paper.