Search Results for author: Sven Behnke

Found 84 papers, 28 papers with code

Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detection

no code implementations2 Feb 2024 Lennard Bodden, Franziska Schwaiger, Duc Bach Ha, Lars Kreuzberg, Sven Behnke

To the best of our knowledge, our work is the first approach that takes advantage of knowledge distillation in the field of spiking object detection.

Knowledge Distillation object-detection +2

Attention-Based VR Facial Animation with Visual Mouth Camera Guidance for Immersive Telepresence Avatars

no code implementations15 Dec 2023 Andre Rochow, Max Schwarz, Sven Behnke

We present a hybrid method that uses both keypoints and direct visual guidance from a mouth camera.

Learning from SAM: Harnessing a Foundation Model for Sim2Real Adaptation by Regularization

no code implementations27 Sep 2023 Mayara E. Bonani, Max Schwarz, Sven Behnke

We present a method for self-supervised domain adaptation for the scenario where annotated source domain data (e. g. from synthetic generation) is available, but the target domain data is completely unannotated.

Domain Adaptation Segmentation +1

The $10 Million ANA Avatar XPRIZE Competition Advanced Immersive Telepresence Systems

no code implementations15 Aug 2023 Sven Behnke, Julie A. Adams, David Locke

The $10M ANA Avatar XPRIZE aimed to create avatar systems that can transport human presence to remote locations in real time.

Learning Generalizable Tool Use with Non-rigid Grasp-pose Registration

no code implementations31 Jul 2023 Malte Mosbach, Sven Behnke

Tool use, a hallmark feature of human intelligence, remains a challenging problem in robotics due the complex contacts and high-dimensional action space.

VR Facial Animation for Immersive Telepresence Avatars

no code implementations24 Apr 2023 Andre Rochow, Max Schwarz, Michael Schreiber, Sven Behnke

In a quick enrollment step, we capture a sequence of source images from the operator without the VR headset which contain all the important operator-specific appearance information.

Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactions

1 code implementation23 Feb 2023 Angel Villar-Corrales, Ismail Wahdan, Sven Behnke

We propose a novel framework for the task of object-centric video prediction, i. e., extracting the compositional structure of a video sequence, as well as modeling objects dynamics and interactions from visual observations in order to predict the future object states, from which we can then generate subsequent video frames.

Object Video Prediction

Rendering the Directional TSDF for Tracking and Multi-Sensor Registration with Point-To-Plane Scale ICP

no code implementations30 Jan 2023 Malte Splietker, Sven Behnke

The recently presented Directional Truncated Signed Distance Function (DTSDF) is an augmentation of the regular TSDF that shows potential for more coherent maps and improved tracking performance.

Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrations

no code implementations5 Dec 2022 Malte Mosbach, Kara Moraw, Sven Behnke

Dexterous manipulation with anthropomorphic robot hands remains a challenging problem in robotics because of the high-dimensional state and action spaces and complex contacts.

Imitation Learning Reinforcement Learning (RL)

PermutoSDF: Fast Multi-View Reconstruction with Implicit Surfaces using Permutohedral Lattices

1 code implementation CVPR 2023 Radu Alexandru Rosu, Sven Behnke

We propose improvements to the two areas by replacing the voxel hash encoding with a permutohedral lattice which optimizes faster, especially for higher dimensions.

Object-level 3D Semantic Mapping using a Network of Smart Edge Sensors

no code implementations21 Nov 2022 Julian Hau, Simon Bultmann, Sven Behnke

Objects are represented in the map via their 3D mesh model or as an object-centric volumetric sub-map that can model arbitrary object geometry when no detailed 3D model is available.

Object Pose Estimation +1

Efficient Representations of Object Geometry for Reinforcement Learning of Interactive Grasping Policies

no code implementations20 Nov 2022 Malte Mosbach, Sven Behnke

Grasping objects of different shapes and sizes - a foundational, effortless skill for humans - remains a challenging task in robotics.

Object reinforcement-learning +1

Online Marker-free Extrinsic Camera Calibration using Person Keypoint Detections

1 code implementation15 Sep 2022 Bastian Pätzold, Simon Bultmann, Sven Behnke

The person keypoint detections from multiple views are received at a central backend where they are synchronized, filtered, and assigned to person hypotheses.

Camera Calibration

Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images

no code implementations28 Jul 2022 Radu Alexandru Rosu, Shunsuke Saito, Ziyan Wang, Chenglei Wu, Sven Behnke, Giljoo Nam

Furthermore, we introduce a novel neural rendering framework based on rasterization of the learned hair strands.

Neural Rendering

3D Semantic Scene Perception using Distributed Smart Edge Sensors

2 code implementations3 May 2022 Simon Bultmann, Sven Behnke

The proposed perception system provides a complete scene view containing semantically annotated 3D geometry and estimates 3D poses of multiple persons in real time.

object-detection Object Detection +2

Abstract Flow for Temporal Semantic Segmentation on the Permutohedral Lattice

1 code implementation29 Mar 2022 Peer Schütt, Radu Alexandru Rosu, Sven Behnke

Semantic segmentation is a core ability required by autonomous agents, as being able to distinguish which parts of the scene belong to which object class is crucial for navigation and interaction with the environment.

Optical Flow Estimation Semantic Segmentation

Synthetic-to-Real Domain Adaptation using Contrastive Unpaired Translation

no code implementations17 Mar 2022 Benedikt T. Imbusch, Max Schwarz, Sven Behnke

We utilize a state-of-the-art image-to-image translation method to adapt the synthetic images to the real domain, minimizing the domain gap in a learned manner.

Domain Adaptation Image-to-Image Translation +1

MSPred: Video Prediction at Multiple Spatio-Temporal Scales with Hierarchical Recurrent Networks

1 code implementation17 Mar 2022 Angel Villar-Corrales, Ani Karapetyan, Andreas Boltres, Sven Behnke

In our experiments, we demonstrate that MSPred accurately predicts future video frames as well as high-level representations (e. g. keypoints or semantics) on bin-picking and action recognition datasets, while consistently outperforming popular approaches for future frame prediction.

 Ranked #1 on Video Prediction on KTH (LPIPS metric)

Video Prediction

Semantic Interaction in Augmented Reality Environments for Microsoft HoloLens

no code implementations18 Nov 2021 Peer Schüett, Max Schwarz, Sven Behnke

We explore this idea using the Microsoft HoloLens, with which we capture indoor environments and display interaction cues with known object classes.

2D Semantic Segmentation Object +1

Fourier-based Video Prediction through Relational Object Motion

no code implementations12 Oct 2021 Malte Mosbach, Sven Behnke

The ability to predict future outcomes conditioned on observed video frames is crucial for intelligent decision-making in autonomous systems.

Decision Making Object +1

Unsupervised Image Decomposition with Phase-Correlation Networks

2 code implementations7 Oct 2021 Angel Villar-Corrales, Sven Behnke

The ability to decompose scenes into their object components is a desired property for autonomous agents, allowing them to reason and act in their surroundings.

Object Object Discovery +2

Rendering and Tracking the Directional TSDF: Modeling Surface Orientation for Coherent Maps

1 code implementation18 Aug 2021 Malte Splietker, Sven Behnke

Dense real-time tracking and mapping from RGB-D images is an important tool for many robotic applications, such as navigation or grasping.

Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles

no code implementations14 Aug 2021 Simon Bultmann, Jan Quenzel, Sven Behnke

In this work, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities.

Image Segmentation object-detection +3

NeuralMVS: Bridging Multi-View Stereo and Novel View Synthesis

1 code implementation9 Aug 2021 Radu Alexandru Rosu, Sven Behnke

Our method uses only a sparse set of images as input and can generalize well to novel scenes.

Novel View Synthesis

SynPick: A Dataset for Dynamic Bin Picking Scene Understanding

1 code implementation10 Jul 2021 Arul Selvam Periyasamy, Max Schwarz, Sven Behnke

We present SynPick, a synthetic dataset for dynamic scene understanding in bin-picking scenarios.

Pose Estimation Scene Understanding

Real-time Pose Estimation from Images for Multiple Humanoid Robots

1 code implementation6 Jul 2021 Arash Amini, Hafez Farazi, Sven Behnke

Pose estimation commonly refers to computer vision methods that recognize people's body postures in images or videos.

Pose Estimation

6D Object Pose Estimation using Keypoints and Part Affinity Fields

no code implementations5 Jul 2021 Moritz Zappel, Simon Bultmann, Sven Behnke

The task of 6D object pose estimation from RGB images is an important requirement for autonomous service robots to be able to interact with the real world.

6D Pose Estimation using RGB Object +1

Real-Time Multi-View 3D Human Pose Estimation using Semantic Feedback to Smart Edge Sensors

1 code implementation28 Jun 2021 Simon Bultmann, Sven Behnke

We present a novel method for estimation of 3D human poses from a multi-camera setup, employing distributed smart edge sensors coupled with a backend through a semantic feedback loop.

3D Multi-Person Pose Estimation

Flexible Table Recognition and Semantic Interpretation System

1 code implementation25 May 2021 Marcin Namysl, Alexander M. Esser, Sven Behnke, Joachim köhler

Moreover, to incorporate the extraction of semantic information, we develop a graph-based table interpretation method.

Table Detection Table Extraction +1

Local Frequency Domain Transformer Networks for Video Prediction

1 code implementation10 May 2021 Hafez Farazi, Jan Nogga, Sven Behnke

Although these models can predict the future frames, they rely entirely on these recurrent structures to simultaneously perform three distinct tasks: extracting transformations, projecting them into the future, and transforming the current frame.

Motion Segmentation Video Prediction

Real-time Multi-Adaptive-Resolution-Surfel 6D LiDAR Odometry using Continuous-time Trajectory Optimization

1 code implementation5 May 2021 Jan Quenzel, Sven Behnke

Simultaneous Localization and Mapping (SLAM) is an essential capability for autonomous robots, but due to high data rates of 3D LiDARs real-time SLAM is challenging.

Simultaneous Localization and Mapping

Robust Skeletonization for Plant Root Structure Reconstruction from MRI

no code implementations27 Oct 2020 Jannis Horn, Yi Zhao, Nils Wandel, Magdalena Landl, Andrea Schnepf, Sven Behnke

Structural reconstruction of plant roots from MRI is challenging, because of low resolution and low signal-to-noise ratio of the 3D measurements which may lead to disconnectivities and wrongly connected roots.

Category-Level 3D Non-Rigid Registration from Single-View RGB Images

no code implementations17 Aug 2020 Diego Rodriguez, Florian Huber, Sven Behnke

This is done by training a CNN that infers a deformation field for the visible parts of the canonical model and by employing a learned shape (latent) space for inferring the deformations of the occluded parts.

NAT: Noise-Aware Training for Robust Neural Sequence Labeling

1 code implementation ACL 2020 Marcin Namysl, Sven Behnke, Joachim köhler

To this end, we formulate the noisy sequence labeling problem, where the input may undergo an unknown noising process and propose two Noise-Aware Training (NAT) objectives that improve robustness of sequence labeling performed on perturbed input: Our data augmentation method trains a neural model using a mixture of clean and noisy samples, whereas our stability training algorithm encourages the model to create a noise-invariant latent representation.

Data Augmentation named-entity-recognition +3

Stillleben: Realistic Scene Synthesis for Deep Learning in Robotics

1 code implementation12 May 2020 Max Schwarz, Sven Behnke

Training data is the key ingredient for deep learning approaches, but difficult to obtain for the specialized domains often encountered in robotics.

object-detection Object Detection +3

Multi-Staged Cross-Lingual Acoustic Model Adaption for Robust Speech Recognition in Real-World Applications - A Case Study on German Oral History Interviews

no code implementations LREC 2020 Michael Gref, Oliver Walter, Christoph Schmidt, Sven Behnke, Joachim K{\"o}hler

While recent automatic speech recognition systems achieve remarkable performance when large amounts of adequate, high quality annotated speech data is used for training, the same systems often only achieve an unsatisfactory result for tasks in domains that greatly deviate from the conditions represented by the training data.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Motion Segmentation using Frequency Domain Transformer Networks

1 code implementation18 Apr 2020 Hafez Farazi, Sven Behnke

Self-supervised prediction is a powerful mechanism to learn representations that capture the underlying structure of the data.

Motion Segmentation Video Prediction

Visual Descriptor Learning from Monocular Video

no code implementations15 Apr 2020 Umashankar Deekshith, Nishit Gajjar, Max Schwarz, Sven Behnke

In this paper, we propose a novel way to estimate dense correspondence on an RGB image where visual descriptors are learned from video examples by training a fully convolutional network.

Optical Flow Estimation

Beyond Photometric Consistency: Gradient-based Dissimilarity for Improving Visual Odometry and Stereo Matching

no code implementations8 Apr 2020 Jan Quenzel, Radu Alexandru Rosu, Thomas Läbe, Cyrill Stachniss, Sven Behnke

We integrate both into stereo estimation as well as visual odometry systems and show clear benefits for typical disparity and direct image registration tasks when using our proposed metric.

Image Registration Pose Estimation +2

3D U-Net for Segmentation of Plant Root MRI Images in Super-Resolution

no code implementations21 Feb 2020 Yi Zhao, Nils Wandel, Magdalena Landl, Andrea Schnepf, Sven Behnke

Magnetic resonance imaging (MRI) enables plant scientists to non-invasively study root system development and root-soil interaction.

Super-Resolution

ConvPoseCNN: Dense Convolutional 6D Object Pose Estimation

no code implementations16 Dec 2019 Catherine Capellen, Max Schwarz, Sven Behnke

Instead we propose pixel-wise, dense prediction of both translation and orientation components of the object pose, where the dense orientation is represented in Quaternion form.

6D Pose Estimation using RGB Clustering +2

RoboCup 2019 AdultSize Winner NimbRo: Deep Learning Perception, In-Walk Kick, Push Recovery, and Team Play Capabilities

2 code implementations16 Dec 2019 Diego Rodriguez, Hafez Farazi, Grzegorz Ficht, Dmytro Pavlichenko, Andre Brandenburger, Mojtaba Hosseini, Oleg Kosenko, Michael Schreiber, Marcel Missura, Sven Behnke

Individual and team capabilities are challenged every year by rule changes and the increasing performance of the soccer teams at RoboCup Humanoid League.

Robotics

Bonn Activity Maps: Dataset Description

no code implementations13 Dec 2019 Julian Tanke, Oh-Hun Kwon, Patrick Stotko, Radu Alexandru Rosu, Michael Weinmann, Hassan Errami, Sven Behnke, Maren Bennewitz, Reinhard Klein, Andreas Weber, Angela Yao, Juergen Gall

The key prerequisite for accessing the huge potential of current machine learning techniques is the availability of large databases that capture the complex relations of interest.

Activity Recognition

NimbRo Robots Winning RoboCup 2018 Humanoid AdultSize Soccer Competitions

1 code implementation5 Sep 2019 Hafez Farazi, Grzegorz Ficht, Philipp Allgeuer, Dmytro Pavlichenko, Diego Rodriguez, Andre Brandenburger, Mojtaba Hosseini, Sven Behnke

Over the past few years, the Humanoid League rules have changed towards more realistic and challenging game environments, which encourage teams to advance their robot soccer performances.

Robotics

Utilizing Temporal Information in Deep Convolutional Network for Efficient Soccer Ball Detection and Tracking

1 code implementation5 Sep 2019 Anna Kukleva, Mohammad Asif Khan, Hafez Farazi, Sven Behnke

We first solve the detection task for an image using fully convolutional encoder-decoder architecture, and later, we use it as an input to our temporal models and jointly learn the detection task in sequences of images.

Game of Football

Two-Staged Acoustic Modeling Adaption for Robust Speech Recognition by the Example of German Oral History Interviews

no code implementations19 Aug 2019 Michael Gref, Christoph Schmidt, Sven Behnke, Joachim köhler

In automatic speech recognition, often little training data is available for specific challenging tasks, but training of state-of-the-art automatic speech recognition systems requires large amounts of annotated speech.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Directional TSDF: Modeling Surface Orientation for Coherent Meshes

no code implementations14 Aug 2019 Malte Splietker, Sven Behnke

Real-time 3D reconstruction from RGB-D sensor data plays an important role in many robotic applications, such as object modeling and mapping.

3D Reconstruction

Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks

no code implementations7 Aug 2019 Jörg Wagner, Jan Mathias Köhler, Tobias Gindele, Leon Hetzel, Jakob Thaddäus Wiedemer, Sven Behnke

Our approach is based on a novel technique to defend against adversarial evidence (i. e. faulty evidence due to artefacts) by filtering gradients during optimization.

valid

Semi-Supervised Semantic Mapping through Label Propagation with Semantic Texture Meshes

no code implementations17 Jun 2019 Radu Alexandru Rosu, Jan Quenzel, Sven Behnke

We propose to represent the semantic map as a geometrical mesh and a semantic texture coupled at independent resolution.

Scene Understanding Semantic Segmentation

Value Iteration Networks on Multiple Levels of Abstraction

1 code implementation27 May 2019 Daniel Schleich, Tobias Klamt, Sven Behnke

In contrast to a multiresolution coarse-to-fine VIN implementation which does not employ additional descriptive features, our approach is capable of solving challenging environments, which demonstrates that the proposed method learns to encode useful information in the additional features.

Descriptive

Learning Super-resolution 3D Segmentation of Plant Root MRI Images from Few Examples

no code implementations16 Mar 2019 Ali Oguz Uzman, Jannis Horn, Sven Behnke

While magnetic resonance imaging (MRI) can be used to obtain 3D images of plant roots, extracting the root structural model is challenging due to highly noisy soil environments and low-resolution of MRI images.

Super-Resolution

Detection and Tracking of Small Objects in Sparse 3D Laser Range Data

no code implementations14 Mar 2019 Jan Razlaw, Jan Quenzel, Sven Behnke

Detection and tracking of dynamic objects is a key feature for autonomous behavior in a continuously changing environment.

Multi-Object Tracking

Complex Valued Gated Auto-encoder for Video Frame Prediction

no code implementations8 Mar 2019 Niloofar Azizi, Nils Wandel, Sven Behnke

Then, we present how a complex neural network can learn such transformations and compare its performance and parameter efficiency to a real-valued gated autoencoder.

Frequency Domain Transformer Networks for Video Prediction

1 code implementation1 Mar 2019 Hafez Farazi, Sven Behnke

The task of video prediction is forecasting the next frames given some previous frames.

Video Prediction

Autonomous Dual-Arm Manipulation of Familiar Objects

no code implementations21 Nov 2018 Dmytro Pavlichenko, Diego Rodriguez, Max Schwarz, Christian Lenz, Arul Selvam Periyasamy, Sven Behnke

The entire pipeline can be executed on-board and is suitable for on-line grasping scenarios.

Robotics

NimbRo-OP2X: Adult-sized Open-source 3D Printed Humanoid Robot

1 code implementation19 Oct 2018 Grzegorz Ficht, Hafez Farazi, André Brandenburger, Diego Rodriguez, Dmytro Pavlichenko, Philipp Allgeuer, Mojtaba Hosseini, Sven Behnke

Humanoid robotics research depends on capable robot platforms, but recently developed advanced platforms are often not available to other research groups, expensive, dangerous to operate, or closed-source.

Robotics

Learning Postural Synergies for Categorical Grasping through Shape Space Registration

no code implementations18 Oct 2018 Diego Rodriguez, Antonio Di Guardo, Antonio Frisoli, Sven Behnke

Grasping knowledge is gathered in a synergy space of the robotic hand built by following a human grasping taxonomy.

Robotics

Real-Time Visual Tracking and Identification for a Team of Homogeneous Humanoid Robots

no code implementations15 Oct 2018 Hafez Farazi, Sven Behnke

The use of a team of humanoid robots to collaborate in completing a task is an increasingly important field of research.

Real-Time Visual Tracking

Online Visual Robot Tracking and Identification using Deep LSTM Networks

no code implementations11 Oct 2018 Hafez Farazi, Sven Behnke

One of the challenges for achieving collaboration in a team of robots is mutual tracking and identification.

Location Dependency in Video Prediction

1 code implementation11 Oct 2018 Niloofar Azizi, Hafez Farazi, Sven Behnke

The task of video prediction requires analyzing the video frames, temporally and spatially, and constructing a model of how the environment evolves.

Video Prediction

Functionally Modular and Interpretable Temporal Filtering for Robust Segmentation

no code implementations9 Oct 2018 Jörg Wagner, Volker Fischer, Michael Herman, Sven Behnke

Our filter module splits the filter task into multiple less complex and more interpretable subtasks.

Hierarchical Recurrent Filtering for Fully Convolutional DenseNets

no code implementations5 Oct 2018 Jörg Wagner, Volker Fischer, Michael Herman, Sven Behnke

Generating a robust representation of the environment is a crucial ability of learning agents.

Transferring Category-based Functional Grasping Skills by Latent Space Non-Rigid Registration

no code implementations14 Sep 2018 Diego Rodriguez, Sven Behnke

Control poses for generating grasping motions are accumulated in the canonical model from grasping definitions of known objects.

Robotics

Learning a Loopy Model For Semantic Segmentation Exactly

no code implementations16 Sep 2013 Andreas Christian Mueller, Sven Behnke

We hope that this insight can lead to a reconsideration of the tractability of loopy models in computer vision.

Semantic Segmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.