Search Results for author: Sebastian Scherer

Found 69 papers, 25 papers with code

Attention-Enhanced Cross-modal Localization Between 360 Images and Point Clouds

no code implementations6 Dec 2022 Zhipeng Zhao, Huai Yu, Chenwei Lyv, Wen Yang, Sebastian Scherer

To overcome this limitation, we focus on correlating the information of 360 equirectangular images to point clouds, proposing an end-to-end learnable network to conduct cross-modal visual localization by establishing similarity in high-dimensional feature space.

Autonomous Driving Camera Localization +1

Pseudo-Label Noise Suppression Techniques for Semi-Supervised Semantic Segmentation

1 code implementation19 Oct 2022 Sebastian Scherer, Robin Schön, Rainer Lienhart

Current SSL approaches use an initially supervised trained model to generate predictions for unlabelled images, called pseudo-labels, which are subsequently used for training a new model from scratch.

Pose Estimation Pseudo Label +1

TartanCalib: Iterative Wide-Angle Lens Calibration using Adaptive SubPixel Refinement of AprilTags

no code implementations5 Oct 2022 Bardienus P Duisterhof, Yaoyu Hu, Si Heng Teng, Michael Kaess, Sebastian Scherer

An accurate calibration of the intrinsics and extrinsics is a critical pre-requisite for using the edge of a wide-angle lens for depth perception and odometry.

360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidance

1 code implementation28 Sep 2022 Shreyas Kulkarni, Peng Yin, Sebastian Scherer

Additionally, we introduce a semantic consistency loss that encourages realistic renderings of novel views.

SSIM

How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversability

no code implementations22 Sep 2022 Mateo Guaman Castro, Samuel Triest, Wenshan Wang, Jason M. Gregory, Felix Sanchez, John G. Rogers III, Sebastian Scherer

Our short-scale navigation results show that using our learned costmaps leads to overall smoother navigation, and provides the robot with a more fine-grained understanding of the robot-terrain interactions.

MUI-TARE: Multi-Agent Cooperative Exploration with Unknown Initial Position

no code implementations22 Sep 2022 Jingtian Yan, Xingqiao Lin, Zhongqiang Ren, Shiqi Zhao, Jieqiong Yu, Chao Cao, Peng Yin, Ji Zhang, Sebastian Scherer

To intelligently balance the robustness of sub-map merging and exploration efficiency, we develop a new approach for lidar-based multi-agent exploration, which can direct one agent to repeat another agent's trajectory in an \emph{adaptive} manner based on the quality indicator of the sub-map merging process.

DytanVO: Joint Refinement of Visual Odometry and Motion Segmentation in Dynamic Environments

2 code implementations17 Sep 2022 Shihao Shen, Yilin Cai, Wenshan Wang, Sebastian Scherer

Learning-based visual odometry (VO) algorithms achieve remarkable performance on common static scenes, benefiting from high-capacity models and massive annotated data, but tend to fail in dynamic, populated environments.

Motion Segmentation Semantic Segmentation +1

iSimLoc: Visual Global Localization for Previously Unseen Environments with Simulated Images

no code implementations14 Sep 2022 Peng Yin, Ivan Cisneros, Ji Zhang, Howie Choset, Sebastian Scherer

The visual camera is an attractive device in beyond visual line of sight (B-VLOS) drone operation, since they are low in size, weight, power, and cost, and can provide redundant modality to GPS failures.

Retrieval Visual Localization

BioSLAM: A Bio-inspired Lifelong Memory System for General Place Recognition

no code implementations30 Aug 2022 Peng Yin, Abulikemu Abuduweili, Shiqi Zhao, Changliu Liu, Sebastian Scherer

We present BioSLAM, a lifelong SLAM framework for learning various new appearances incrementally and maintaining accurate place recognition for previously visited areas.

ALTO: A Large-Scale Dataset for UAV Visual Place Recognition and Localization

1 code implementation19 Jul 2022 Ivan Cisneros, Peng Yin, Ji Zhang, Howie Choset, Sebastian Scherer

We present the ALTO dataset, a vision-focused dataset for the development and benchmarking of Visual Place Recognition and Localization methods for Unmanned Aerial Vehicles.

Benchmarking Image Registration +2

SphereVLAD++: Attention-based and Signal-enhanced Viewpoint Invariant Descriptor

no code implementations6 Jul 2022 Shiqi Zhao, Peng Yin, Ge Yi, Sebastian Scherer

Our previous work provides a viewpoint-invariant descriptor to deal with viewpoint differences; however, the global descriptor suffers from a low signal-noise ratio in unsupervised clustering, reducing the distinguishable feature extraction ability.

Autonomous Driving Retrieval

Detection and Physical Interaction with Deformable Linear Objects

no code implementations17 May 2022 Azarakhsh Keipour, Mohammadreza Mousaei, Maryam Bandari, Stefan Schaal, Sebastian Scherer

On the other hand, while physical interaction with these objects has been done with ground manipulators, there have not been any studies on physical interaction and manipulation of the deformable linear object with aerial robots.

Off-Policy Evaluation with Online Adaptation for Robot Exploration in Challenging Environments

no code implementations7 Apr 2022 Yafei Hu, Junyi Geng, Chen Wang, John Keller, Sebastian Scherer

Results demonstrate that our method enables the robot to predict the value of future states so as to better guide robot exploration.

Off-policy evaluation

AirObject: A Temporally Evolving Graph Embedding for Object Identification

1 code implementation CVPR 2022 Nikhil Varma Keetha, Chen Wang, Yuheng Qiu, Kuan Xu, Sebastian Scherer

In this context, we propose a novel temporal 3D object encoding approach, dubbed AirObject, to obtain global keypoint graph-based embeddings of objects.

Graph Attention Graph Embedding +1

AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition

no code implementations23 Nov 2021 Haowen Lai, Peng Yin, Sebastian Scherer

Recent years have witnessed the increasing application of place recognition in various environments, such as city roads, large buildings, and a mix of indoor and outdoor places.

Drone flight data reveal energy and greenhouse gas emissions savings for small package delivery

no code implementations22 Nov 2021 Thiago A. Rodrigues, Jay Patrikar, Natalia L. Oliveira, H. Scott Matthews, Sebastian Scherer, Constantine Samaras

The adoption of Uncrewed Aerial Vehicles (UAVs) for last-mile deliveries will affect the energy productivity of package delivery and require new methods to understand the associated energy consumption and greenhouse gas (GHG) emissions.

Unsupervised Online Learning for Robotic Interestingness with Visual Memory

1 code implementation18 Nov 2021 Chen Wang, Yuheng Qiu, Wenshan Wang, Yafei Hu, Seungchan Kim, Sebastian Scherer

Instead, we propose to develop a method that automatically adapts online to the environment to report interesting scenes quickly.

Translation

Semantically Consistent Image-to-Image Translation for Unsupervised Domain Adaptation

no code implementations5 Nov 2021 Stephan Brehm, Sebastian Scherer, Rainer Lienhart

Unsupervised Domain Adaptation (UDA) aims to adapt models trained on a source domain to a new target domain where no labelled data is available.

Image-to-Image Translation Semantic Segmentation +2

AirDOS: Dynamic SLAM benefits from Articulated Objects

1 code implementation21 Sep 2021 Yuheng Qiu, Chen Wang, Wenshan Wang, Mina Henein, Sebastian Scherer

To the best of our knowledge, AirDOS is the first dynamic object-aware SLAM system demonstrating that camera pose estimation can be improved by incorporating dynamic articulated objects.

Motion Estimation Pose Estimation

AirLoop: Lifelong Loop Closure Detection

1 code implementation18 Sep 2021 Dasong Gao, Chen Wang, Sebastian Scherer

Nevertheless, simply finetuning the model on new data is infeasible since it may cause the model's performance on previously learned data to degrade over time, which is also known as the problem of catastrophic forgetting.

Incremental Learning Loop Closure Detection +1

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

no code implementations12 Aug 2021 Zheng Fang, Sifan Zhou, Yubo Cui, Sebastian Scherer

Then, to fuse the information of features in the two branches and obtain their similarity, we propose two cross correlation modules, named Pointcloud-wise and Point-wise respectively.

3D Single Object Tracking Object Tracking +1

3D Human Reconstruction in the Wild with Collaborative Aerial Cameras

no code implementations9 Aug 2021 Cherie Ho, Andrew Jong, Harry Freeman, Rohan Rao, Rogerio Bonatti, Sebastian Scherer

Aerial vehicles are revolutionizing applications that require capturing the 3D structure of dynamic targets in the wild, such as sports, medicine, and entertainment.

3D Human Reconstruction

i3dLoc: Image-to-range Cross-domain Localization Robust to Inconsistent Environmental Conditions

no code implementations27 May 2021 Peng Yin, Lingyun Xu, Ji Zhang, Howie Choset, Sebastian Scherer

Based on such features, we further design a spherical convolution network to learn viewpoint-invariant symmetric place descriptors.

Visual Localization

3D Segmentation Learning from Sparse Annotations and Hierarchical Descriptors

no code implementations27 May 2021 Peng Yin, Lingyun Xu, Jianmin Ji, Sebastian Scherer, Howie Choset

One of the main obstacles to 3D semantic segmentation is the significant amount of endeavor required to generate expensive point-wise annotations for fully supervised training.

3D Semantic Segmentation

ORStereo: Occlusion-Aware Recurrent Stereo Matching for 4K-Resolution Images

no code implementations13 Mar 2021 Yaoyu Hu, Wenshan Wang, Huai Yu, Weikun Zhen, Sebastian Scherer

Stereo reconstruction models trained on small images do not generalize well to high-resolution data.

Stereo Matching

Real-Time Ellipse Detection for Robotics Applications

no code implementations25 Feb 2021 Azarakhsh Keipour, Guilherme A. S. Pereira, Sebastian Scherer

We propose a new algorithm for real-time detection and tracking of elliptic patterns suitable for real-world robotics applications.

Bridging Graph Network to Lifelong Learning with Feature Interaction

no code implementations1 Jan 2021 Chen Wang, Yuheng Qiu, Sebastian Scherer

In this paper, we aim to bridge GNN to lifelong learning, which is to overcome the effect of ``catastrophic forgetting" for continuously learning a sequence of graph-structured tasks.

Graph Classification Node Classification

Batteries, camera, action! Learning a semantic control space for expressive robot cinematography

no code implementations19 Nov 2020 Rogerio Bonatti, Arthur Bucker, Sebastian Scherer, Mustafa Mukadam, Jessica Hodgins

First, we generate a database of video clips with a diverse range of shots in a photo-realistic simulator, and use hundreds of participants in a crowd-sourcing framework to obtain scores for a set of semantic descriptors for each clip.

Integration of Fully-Actuated Multirotors into Real-World Applications

no code implementations12 Nov 2020 Azarakhsh Keipour, Mohammadreza Mousaei, Andrew T Ashley, Sebastian Scherer

The strategies are lightweight, simple, and allow rapid integration of the available tools with these new vehicles for the fast development of new real-world applications.

Do You See What I See? Coordinating Multiple Aerial Cameras for Robot Cinematography

no code implementations10 Nov 2020 Arthur Bucker, Rogerio Bonatti, Sebastian Scherer

We validate our approach in multiple cluttered environments of a photo-realistic simulator, and deploy the system using two UAVs in real-world experiments.

ULSD: Unified Line Segment Detection across Pinhole, Fisheye, and Spherical Cameras

2 code implementations6 Nov 2020 Hao Li, Huai Yu, Wen Yang, Lei Yu, Sebastian Scherer

Targeting at the unified line segment detection (ULSD) for both distorted and undistorted images, we propose to represent line segments with the Bezier curve model.

Line Segment Detection

TartanVO: A Generalizable Learning-based VO

2 code implementations31 Oct 2020 Wenshan Wang, Yaoyu Hu, Sebastian Scherer

We present the first learning-based visual odometry (VO) model, which generalizes to multiple datasets and real-world scenarios and outperforms geometry-based methods in challenging scenes.

Visual Odometry

Lifelong Graph Learning

2 code implementations CVPR 2022 Chen Wang, Yuheng Qiu, Dasong Gao, Sebastian Scherer

In this paper, we bridge GNN and lifelong learning by converting a continual graph learning problem to a regular graph learning problem so GNN can inherit the lifelong learning techniques developed for convolutional neural networks (CNN).

Action Recognition Continual Learning +4

TartanAir: A Dataset to Push the Limits of Visual SLAM

1 code implementation31 Mar 2020 Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, Sebastian Scherer

We present a challenging dataset, the TartanAir, for robot navigation task and more.

Robotics

Deep-Learning Assisted High-Resolution Binocular Stereo Depth Reconstruction

no code implementations23 Nov 2019 Yaoyu Hu, Weikun Zhen, Sebastian Scherer

The proposed deep-learning assisted method is evaluated on the Middlebury dataset and high-resolution stereo images collected by our customized binocular stereo camera.

3D Reconstruction

Line-based Camera Pose Estimation in Point Cloud of Structured Environments

no code implementations23 Nov 2019 Huai Yu, Weikun Zhen, Wen Yang, Sebastian Scherer

In this paper, we propose a new 2D-3D registration method to estimate 2D-3D line feature correspondences and the camera pose in untextured point clouds of structured environments.

Camera Localization Pose Estimation +2

Autonomous Aerial Cinematography In Unstructured Environments With Learned Artistic Decision-Making

no code implementations15 Oct 2019 Rogerio Bonatti, Wenshan Wang, Cherie Ho, Aayush Ahuja, Mirko Gschwindt, Efe Camci, Erdal Kayacan, Sanjiban Choudhury, Sebastian Scherer

In this work, we address the problem in its entirety and propose a complete system for real-time aerial cinematography that for the first time combines: (1) vision-based target estimation; (2) 3D signed-distance mapping for occlusion estimation; (3) efficient trajectory optimization for long time-horizon camera motion; and (4) learning-based artistic shot selection.

Decision Making Occlusion Estimation

A Stereo Algorithm for Thin Obstacles and Reflective Objects

no code implementations3 Oct 2019 John Keller, Sebastian Scherer

We present a hierarchical disparity algorithm that reduces noise, separately identify wires using semantic object triangulation in three images, and use information about the polarization of light to estimate the disparity of reflective objects.

Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations

2 code implementations16 Sep 2019 Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor

We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors.

Drone navigation Imitation Learning

ALFA: A Dataset for UAV Fault and Anomaly Detection

2 code implementations14 Jul 2019 Azarakhsh Keipour, Mohammadreza Mousaei, Sebastian Scherer

We have also provided the helper tools in several programming languages to load and work with the data and to help the evaluation of a detection method using the dataset.

Anomaly Detection Fault Detection

Automatic Real-time Anomaly Detection for Autonomous Aerial Vehicles

2 code implementations1 Jul 2019 Azarakhsh Keipour, Mohammadreza Mousaei, Sebastian Scherer

The result is an easy-to-deploy anomaly detection method that does not assume a specific aircraft model and can detect many types of faults and anomalies in a wide range of autonomous aircraft.

Systems and Control Robotics Systems and Control

Towards a Robust Aerial Cinematography Platform: Localizing and Tracking Moving Targets in Unstructured Environments

no code implementations4 Apr 2019 Rogerio Bonatti, Cherie Ho, Wenshan Wang, Sanjiban Choudhury, Sebastian Scherer

In this work, we overcome such limitations and propose a complete system for aerial cinematography that combines: (1) a vision-based algorithm for target localization; (2) a real-time incremental 3D signed-distance map algorithm for occlusion and safety computation; and (3) a real-time camera motion planner that optimizes smoothness, collisions, occlusions and artistic guidelines.

Pose Estimation

Improved Generalization of Heading Direction Estimation for Aerial Filming Using Semi-supervised Regression

no code implementations26 Mar 2019 Wenshan Wang, Aayush Ahuja, Yanfu Zhang, Rogerio Bonatti, Sebastian Scherer

We show that by leveraging unlabeled sequences, the amount of labeled data required can be significantly reduced.

regression

RGB-D SLAM in Dynamic Environments Using Point Correlations

no code implementations8 Nov 2018 Weichen Dai, Yu Zhang, Ping Li, Zheng Fang, Sebastian Scherer

This method utilizes the correlation between map points to separate points that are part of the static scene and points that are part of different moving objects into different groups.

Motion Estimation Simultaneous Localization and Mapping

Joint Point Cloud and Image Based Localization For Efficient Inspection in Mixed Reality

1 code implementation5 Nov 2018 Manash Pratim Das, Zhen Dong, Sebastian Scherer

While external pose estimation and fiducial marker based localization would require setup, maintenance, and manual calibration; marker-free self-localization can be achieved using the onboard depth sensor and camera.

Robotics

Integrating kinematics and environment context into deep inverse reinforcement learning for predicting off-road vehicle trajectories

1 code implementation16 Oct 2018 Yanfu Zhang, Wenshan Wang, Rogerio Bonatti, Daniel Maturana, Sebastian Scherer

The first-stage network learns feature representations of the environment using low-level LiDAR statistics and the second-stage network combines those learned features with kinematics data.

Autonomous Navigation motion prediction +1

Monocular Object and Plane SLAM in Structured Environments

no code implementations10 Sep 2018 Shichao Yang, Sebastian Scherer

In this paper, we present a monocular Simultaneous Localization and Mapping (SLAM) algorithm using high-level object and plane landmarks.

Camera Localization Simultaneous Localization and Mapping

Autonomous drone cinematographer: Using artistic principles to create smooth, safe, occlusion-free trajectories for aerial filming

no code implementations28 Aug 2018 Rogerio Bonatti, yanfu Zhang, Sanjiban Choudhury, Wenshan Wang, Sebastian Scherer

Autonomous aerial cinematography has the potential to enable automatic capture of aesthetically pleasing videos without requiring human intervention, empowering individuals with the capability of high-end film studios.

CubeSLAM: Monocular 3D Object SLAM

5 code implementations1 Jun 2018 Shichao Yang, Sebastian Scherer

Objects can provide long-range geometric and scale constraints to improve camera pose estimation and reduce monocular drift.

3D Object Detection object-detection +2

Hindsight is Only 50/50: Unsuitability of MDP based Approximate POMDP Solvers for Multi-resolution Information Gathering

no code implementations7 Apr 2018 Sankalp Arora, Sanjiban Choudhury, Sebastian Scherer

The contribution of the paper helps identify the properties of a POMDP problem for which the use of MDP based POMDP solvers is inappropriate, enabling better design choices.

Imitation Learning

Bayesian Active Edge Evaluation on Expensive Graphs

no code implementations20 Nov 2017 Sanjiban Choudhury, Siddhartha Srinivasa, Sebastian Scherer

We are interested in planning algorithms that actively infer the underlying structure of the valid configuration space during planning in order to find solutions with minimal effort.

Active Learning Motion Planning

Improving Stochastic Policy Gradients in Continuous Control with Deep Reinforcement Learning using the Beta Distribution

no code implementations ICML 2017 Po-Wei Chou, Daniel Maturana, Sebastian Scherer

Recently, reinforcement learning with deep neural networks has achieved great success in challenging continuous control problems such as 3D locomotion and robotic manipulation.

Continuous Control

Learning Heuristic Search via Imitation

1 code implementation10 Jul 2017 Mohak Bhardwaj, Sanjiban Choudhury, Sebastian Scherer

In this paper, we do so by training a heuristic policy that maps the partial information from the search to decide which node of the search tree to expand.

Motion Planning

Near-Optimal Edge Evaluation in Explicit Generalized Binomial Graphs

1 code implementation NeurIPS 2017 Sanjiban Choudhury, Shervin Javdani, Siddhartha Srinivasa, Sebastian Scherer

By leveraging this property, we are able to significantly reduce computational complexity from exponential to linear in the number of edges.

Robotics

Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments

no code implementations21 Mar 2017 Shichao Yang, Yu Song, Michael Kaess, Sebastian Scherer

In this paper, we propose real-time monocular plane SLAM to demonstrate that scene understanding could improve both state estimation and dense mapping especially in low-texture environments.

Motion Planning Scene Understanding +1

Direct Monocular Odometry Using Points and Lines

no code implementations19 Mar 2017 Shichao Yang, Sebastian Scherer

Most visual odometry algorithm for a monocular camera focuses on points, either by feature matching, or direct alignment of pixel intensity, while ignoring a common but important geometry entity: edges.

Stereo Matching Stereo Matching Hand +1

Cannot find the paper you are looking for? You can Submit a new open access paper.