Search Results for author: Nassir Navab

Found 276 papers, 77 papers with code

Towards Autonomous Atlas-based Ultrasound Acquisitions in Presence of Articulated Motion

1 code implementation10 Aug 2022 Zhongliang Jiang, Yuan Gao, Le Xie, Nassir Navab

Robotic ultrasound (US) imaging aims at overcoming some of the limitations of free-hand US examinations, e. g. difficulty in guaranteeing intra- and inter-operator repeatability.

Speckle2Speckle: Unsupervised Learning of Ultrasound Speckle Filtering Without Clean Data

1 code implementation31 Jul 2022 Rüdiger Göbl, Christoph Hennersperger, Nassir Navab

To enable this, we make use of realistic ultrasound simulation techniques that allow for instantiation of several independent speckle realizations that represent the exact same tissue, thus allowing for the application of image reconstruction techniques that work with pairs of differently corrupted data.

Image Reconstruction

DA$^2$ Dataset: Toward Dexterity-Aware Dual-Arm Grasping

no code implementations31 Jul 2022 Guangyao Zhai, Yu Zheng, Ziwei Xu, Xin Kong, Yong liu, Benjamin Busam, Yi Ren, Nassir Navab, Zhengyou Zhang

In this paper, we introduce DA$^2$, the first large-scale dual-arm dexterity-aware dataset for the generation of optimal bimanual grasping pairs for arbitrary large objects.

CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning

no code implementations31 Jul 2022 Mahdi Saleh, Yige Wang, Nassir Navab, Benjamin Busam, Federico Tombari

The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods while requiring significantly fewer computations.

Scene Segmentation

RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation

1 code implementation30 Jul 2022 Ruida Zhang, Yan Di, Zhiqiang Lou, Fabian Manhardt, Nassir Navab, Federico Tombari, Xiangyang Ji

Category-level object pose estimation aims to predict the 6D pose as well as the 3D metric size of arbitrary objects from a known set of categories.

Pose Estimation

Spotlight on nerves: Portable multispectral optoacoustic imaging of peripheral nerve vascularization and morphology

no code implementations28 Jul 2022 Dominik Jüstel, Hedwig Irl, Florian Hinterwimmer, Christoph Dehner, Walter Simson, Nassir Navab, Gerhard Schneider, Vasilis Ntziachristos

Various morphological and functional parameters of peripheral nerves and their vascular supply are indicative of pathological changes due to injury or disease.

Exploiting Diversity of Unlabeled Data for Label-Efficient Semi-Supervised Active Learning

no code implementations25 Jul 2022 Felix Buchert, Nassir Navab, Seong Tae Kim

By considering the consistency information with the diversity in the consistency-based embedding scheme, the proposed method could select more informative samples for labeling in the semi-supervised learning setting.

Active Learning Representation Learning

Unsupervised pre-training of graph transformers on patient population graphs

1 code implementation21 Jul 2022 Chantal Pellegrini, Nassir Navab, Anees Kazi

We find that our proposed pre-training methods help in modeling the data at a patient and population level and improve performance in different fine-tuning tasks on all datasets.

Language Modelling Masked Language Modeling +3

CACTUSS: Common Anatomical CT-US Space for US examinations

1 code implementation18 Jul 2022 Yordanka Velikova, Walter Simson, Mehrdad Salehi, Mohammad Farid Azampour, Philipp Paprottka, Nassir Navab

Abdominal aortic aneurysm (AAA) is a vascular disease in which a section of the aorta enlarges, weakening its walls and potentially rupturing the vessel.

Image-to-Image Translation

CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN

1 code implementation15 Jul 2022 Matan Atad, Vitalii Dmytrenko, Yitong Li, Xinyue Zhang, Matthias Keicher, Jan Kirschke, Bene Wiestler, Ashkan Khakzar, Nassir Navab

Deep learning models used in medical image analysis are prone to raising reliability concerns due to their black-box nature.

Shape-Aware Masking for Inpainting in Medical Imaging

no code implementations12 Jul 2022 Yousef Yeganeh, Azade Farshad, Nassir Navab

Inpainting has recently been proposed as a successful deep learning technique for unsupervised medical image model discovery.

Anatomy Image Reconstruction +1

Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data

no code implementations7 Jul 2022 Yousef Yeganeh, Azade Farshad, Johann Boschmann, Richard Gaus, Maximilian Frantzen, Nassir Navab

Although most medical centers conduct similar medical imaging tasks, their differences, such as specializations, number of patients, and devices, lead to distinctive data distributions.

Federated Learning Lesion Classification +2

Unsupervised Cross-Domain Feature Extraction for Single Blood Cell Image Classification

no code implementations1 Jul 2022 Raheleh Salehi, Ario Sadafi, Armin Gruber, Peter Lienemann, Nassir Navab, Shadi Albarqouni, Carsten Marr

Here, we propose a cross-domain adapted autoencoder to extract features in an unsupervised manner on three different datasets of single white blood cells scanned from peripheral blood smears.

Image Classification

DeStripe: A Self2Self Spatio-Spectral Graph Neural Network with Unfolded Hessian for Stripe Artifact Removal in Light-sheet Microscopy

no code implementations27 Jun 2022 Yu Liu, Kurt Weiss, Nassir Navab, Carsten Marr, Jan Huisken, Tingying Peng

Light-sheet fluorescence microscopy (LSFM) is a cutting-edge volumetric imaging technique that allows for three-dimensional imaging of mesoscopic samples with decoupled illumination and detection paths.


U-PET: MRI-based Dementia Detection with Joint Generation of Synthetic FDG-PET Images

no code implementations16 Jun 2022 Marcel Kollovieh, Matthias Keicher, Stephan Wunderlich, Hendrik Burwinkel, Thomas Wendler, Nassir Navab

To this end, we propose a multi-task method based on U-Net that takes T1-weighted MR images as an input to generate synthetic FDG-PET images and classifies the dementia progression of the patient into cognitive normal (CN), cognitive impairment (MCI), and AD.

Virtual embeddings and self-consistency for self-supervised learning

no code implementations13 Jun 2022 Tariq Bdair, Hossam Abdelhamid, Nassir Navab, Shadi Albarqouni

We validate TriMix on eight benchmark datasets consisting of natural and medical images with an improvement of 2. 71% and 0. 41% better than the second-best models for both data types.

Data Augmentation Representation Learning +1

BFS-Net: Weakly Supervised Cell Instance Segmentation from Bright-Field Microscopy Z-Stacks

no code implementations9 Jun 2022 Shervin Dehghani, Benjamin Busam, Nassir Navab, Ali Nasseri

Despite its broad availability, volumetric information acquisition from Bright-Field Microscopy (BFM) is inherently difficult due to the projective nature of the acquisition process.

Instance Segmentation Semantic Segmentation

VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation

1 code implementation10 May 2022 Yuan Bi, Zhongliang Jiang, Yuan Gao, Thomas Wendler, Angelos Karlas, Nassir Navab

The results demonstrate that proposed approach can effectively and accurately navigate the probe towards the longitudinal view of vessels.

Navigate reinforcement-learning

Affective Medical Estimation and Decision Making via Visualized Learning and Deep Learning

1 code implementation9 May 2022 Mohammad Eslami, Solale Tabarestani, Ehsan Adeli, Glyn Elwyn, Tobias Elze, Mengyu Wang, Nazlee Zebardast, Nassir Navab, Malek Adjouadi

With the advent of sophisticated machine learning (ML) techniques and the promising results they yield, especially in medical applications, where they have been investigated for different tasks to enhance the decision-making process.

Decision Making

Y-Net: A Spatiospectral Dual-Encoder Networkfor Medical Image Segmentation

1 code implementation15 Apr 2022 Azade Farshad, Yousef Yeganeh, Peter Gehlbach, Nassir Navab

Automated segmentation of retinal optical coherence tomography (OCT) images has become an important recent direction in machine learning for medical applications.

Medical Image Segmentation Retinal OCT Layer Segmentation +1

Analyzing the Effects of Handling Data Imbalance on Learned Features from Medical Images by Looking Into the Models

no code implementations4 Apr 2022 Ashkan Khakzar, Yawei Li, Yang Zhang, Mirac Sanisoglu, Seong Tae Kim, Mina Rezaei, Bernd Bischl, Nassir Navab

One challenging property lurking in medical datasets is the imbalanced data distribution, where the frequency of the samples between the different classes is not balanced.

Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications

no code implementations1 Apr 2022 Kamilia Mullakaeva, Luca Cosmo, Anees Kazi, Seyed-Ahmad Ahmadi, Nassir Navab, Michael M. Bronstein

In this work, we propose Graph-in-Graph (GiG), a neural network architecture for protein classification and brain imaging applications that exploits the graph representation of the input data samples and their latent relation.

Intelligent Masking: Deep Q-Learning for Context Encoding in Medical Image Analysis

no code implementations25 Mar 2022 Mojtaba Bahrami, Mahsa Ghorbani, Nassir Navab

We show that training the agent against the prediction model can significantly improve the semantic features extracted for downstream classification tasks.

Q-Learning Self-Supervised Learning

Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions

2 code implementations23 Mar 2022 Chantal Pellegrini, Anees Kazi, Nassir Navab

We test our method on two medical datasets of patient records, TADPOLE and MIMIC-III, including imaging and non-imaging features and different prediction tasks.

Disease Prediction Imputation +3

Conditional Generative Data Augmentation for Clinical Audio Datasets

no code implementations22 Mar 2022 Matthias Seibold, Armando Hoch, Mazda Farshad, Nassir Navab, Philipp Fürnstahl

In this work, we propose a novel data augmentation method for clinical audio datasets based on a conditional Wasserstein Generative Adversarial Network with Gradient Penalty (cWGAN-GP), operating on log-mel spectrograms.

Data Augmentation

4D-OR: Semantic Scene Graphs for OR Domain Modeling

no code implementations22 Mar 2022 Ege Özsoy, Evin Pınar Örnek, Ulrich Eck, Tobias Czempiel, Federico Tombari, Nassir Navab

Towards this goal, for the first time, we propose using semantic scene graphs (SSG) to describe and summarize the surgical scene.

Scene Graph Generation

Know your sensORs -- A Modality Study For Surgical Action Classification

no code implementations16 Mar 2022 Lennart Bastian, Tobias Czempiel, Christian Heiliger, Konrad Karcz, Ulrich Eck, Benjamin Busam, Nassir Navab

Existing datasets from OR room cameras are thus far limited in size or modalities acquired, leaving it unclear which sensor modalities are best suited for tasks such as recognizing surgical action from videos.

Action Classification Action Recognition +1

GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting

2 code implementations CVPR 2022 Yan Di, Ruida Zhang, Zhiqiang Lou, Fabian Manhardt, Xiangyang Ji, Nassir Navab, Federico Tombari

While 6D object pose estimation has recently made a huge leap forward, most methods can still only handle a single or a handful of different objects, which limits their applications.

 Ranked #1 on 6D Pose Estimation on LineMOD (Mean ADD-S metric)

6D Pose Estimation 6D Pose Estimation using RGB +1

From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Prediction

no code implementations15 Mar 2022 Evin Pınar Örnek, Shristi Mudgal, Johanna Wald, Yida Wang, Nassir Navab, Federico Tombari

There have been numerous recently proposed methods for monocular depth prediction (MDP) coupled with the equally rapid evolution of benchmarking tools.

Depth Estimation Depth Prediction

Transformers in Action: Weakly Supervised Action Segmentation

no code implementations14 Jan 2022 John Ridley, Huseyin Coskun, David Joseph Tan, Nassir Navab, Federico Tombari

The video action segmentation task is regularly explored under weaker forms of supervision, such as transcript supervision, where a list of actions is easier to obtain than dense frame-wise labels.

Action Segmentation

A Variational Bayesian Method for Similarity Learning in Non-Rigid Image Registration

1 code implementation CVPR 2022 Daniel Grzech, Mohammad Farid Azampour, Ben Glocker, Julia Schnabel, Nassir Navab, Bernhard Kainz, Loïc le Folgoc

We propose a novel variational Bayesian formulation for diffeomorphic non-rigid registration of medical images, which learns in an unsupervised way a data-specific similarity metric.

Image Registration

Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments

no code implementations7 Dec 2021 HyunJun Jung, Nikolas Brasch, Ales Leonardis, Nassir Navab, Benjamin Busam

Indirect Time-of-Flight (I-ToF) imaging is a widespread way of depth estimation for mobile devices due to its small size and affordable price.

Depth Estimation Depth Prediction

Object-aware Monocular Depth Prediction with Instance Convolutions

1 code implementation2 Dec 2021 Enis Simsar, Evin Pınar Örnek, Fabian Manhardt, Helisa Dhamo, Nassir Navab, Federico Tombari

With the advent of deep learning, estimating depth from a single RGB image has recently received a lot of attention, being capable of empowering many different applications ranging from path planning for robotics to computational cinematography.

Depth Estimation Depth Prediction +1

MIGS: Meta Image Generation from Scene Graphs

1 code implementation22 Oct 2021 Azade Farshad, Sabrina Musatian, Helisa Dhamo, Nassir Navab

We propose MIGS (Meta Image Generation from Scene Graphs), a meta-learning based approach for few-shot image generation from graphs that enables adapting the model to different scenes and increases the image quality by training on diverse sets of tasks.

Image Generation from Scene Graphs Meta-Learning +1

Semantic Image Alignment for Vehicle Localization

no code implementations8 Oct 2021 Markus Herb, Matthias Lemberger, Marcel M. Schmitt, Alexander Kurz, Tobias Weiherer, Nassir Navab, Federico Tombari

Accurate and reliable localization is a fundamental requirement for autonomous vehicles to use map information in higher-level tasks such as navigation or planning.

Autonomous Vehicles Semantic Segmentation +1

Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation

no code implementations24 Sep 2021 Mert Asim Karaoglu, Nikolas Brasch, Marijn Stollenga, Wolfgang Wein, Nassir Navab, Federico Tombari, Alexander Ladikos

The results of our experiments show that the proposed method improves the network's performance on real images by a considerable margin and can be employed in 3D reconstruction pipelines.

3D Reconstruction Depth Estimation

MetaMedSeg: Volumetric Meta-learning for Few-Shot Organ Segmentation

no code implementations18 Sep 2021 Anastasia Makarevich, Azade Farshad, Vasileios Belagiannis, Nassir Navab

In this work, we present MetaMedSeg, a gradient-based meta-learning algorithm that redefines the meta-learning task for the volumetric medical data with the goal to capture the variety between the slices.

Medical Image Segmentation Meta-Learning +1

Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs

1 code implementation ICCV 2021 Helisa Dhamo, Fabian Manhardt, Nassir Navab, Federico Tombari

Scene graphs are representations of a scene, composed of objects (nodes) and inter-object relationships (edges), proven to be particularly suited for this task, as they allow for semantic control on the generated content.

SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

2 code implementations ICCV 2021 Yan Di, Fabian Manhardt, Gu Wang, Xiangyang Ji, Nassir Navab, Federico Tombari

Directly regressing all 6 degrees-of-freedom (6DoF) for the object pose (e. g. the 3D rotation and translation) in a cluttered environment from a single RGB image is a challenging problem.

6D Pose Estimation 6D Pose Estimation using RGB +1

Unconditional Scene Graph Generation

no code implementations ICCV 2021 Sarthak Garg, Helisa Dhamo, Azade Farshad, Sabrina Musatian, Nassir Navab, Federico Tombari

Scene graphs, composed of nodes as objects and directed-edges as relationships among objects, offer an alternative representation of a scene that is more semantically grounded than images.

Anomaly Detection Graph Generation +2

Tracked 3D Ultrasound and Deep Neural Network-based Thyroid Segmentation reduce Interobserver Variability in Thyroid Volumetry

no code implementations10 Aug 2021 Markus Krönke, Christine Eilers, Desislava Dimova, Melanie Köhler, Gabriel Buschner, Lilit Mirzojan, Lemonia Konstantinidou, Marcus R. Makowski, James Nagarajah, Nassir Navab, Wolfgang Weber, Thomas Wendler

Conclusion: Tracked 3D ultrasound combined with a CNN segmentation significantly reduces interobserver variability in thyroid volumetry and increases the accuracy of the measurements with shorter acquisition times.

R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes

no code implementations10 Aug 2021 Stefano Gasperini, Patrick Koch, Vinzenz Dallabetta, Nassir Navab, Benjamin Busam, Federico Tombari

While self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches, violations of the static world assumption can still lead to erroneous depth predictions of traffic participants, posing a potential safety issue.

Autonomous Vehicles Monocular Depth Estimation

U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction

no code implementations29 Jul 2021 Matthias Keicher, Hendrik Burwinkel, David Bani-Harouni, Magdalini Paschali, Tobias Czempiel, Egon Burian, Marcus R. Makowski, Rickmer Braren, Nassir Navab, Thomas Wendler

Specifically, we introduce a multimodal similarity metric to build a population graph for clustering patients and an image-based end-to-end Graph Attention Network to process this graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation and mortality.

Decision Making Graph Attention

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

2 code implementations26 Jul 2021 Daniil Pakhomov, Sanchit Hira, Narayani Wagle, Kemar E. Green, Nassir Navab

Derived regions are consistent across different images and coincide with human-defined semantic classes on some datasets.

Semantic Segmentation

Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images

no code implementations9 Jun 2021 Jakob Weiss, Nassir Navab

In this work, we introduce Deep Direct Volume Rendering (DeepDVR), a generalization of DVR that allows for the integration of deep neural networks into the DVR algorithm.

Colorization Neural Rendering

Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedures

no code implementations9 Jun 2021 Ege Özsoy, Evin Pınar Örnek, Ulrich Eck, Federico Tombari, Nassir Navab

We then use MSSG to introduce a dynamically generated graphical user interface tool for surgical procedure analysis which could be used for many applications including process optimization, OR design and automatic report generation.

GKD: Semi-supervised Graph Knowledge Distillation for Graph-Independent Inference

no code implementations8 Apr 2021 Mahsa Ghorbani, Mojtaba Bahrami, Anees Kazi, Mahdieh SoleymaniBaghshah, Hamid R. Rabiee, Nassir Navab

The soft pseudo-labels are then used to train a deep student network for disease prediction of unseen test data for which the graph modality is unavailable.

Disease Prediction graph construction +1

Towards Semantic Interpretation of Thoracic Disease and COVID-19 Diagnosis Models

1 code implementation4 Apr 2021 Ashkan Khakzar, Sabrina Musatian, Jonas Buchberger, Icxel Valeriano Quiroz, Nikolaus Pinger, Soroosh Baselizadeh, Seong Tae Kim, Nassir Navab

We present our findings using publicly available chest pathologies (CheXpert, NIH ChestX-ray8) and COVID-19 datasets (BrixIA, and COVID-19 chest X-ray segmentation dataset).

COVID-19 Diagnosis

IA-GCN: Interpretable Attention based Graph Convolutional Network for Disease prediction

no code implementations29 Mar 2021 Anees Kazi, Soroush Farghadani, Nassir Navab

The main novelty lies in the interpretable attention module (IAM), which directly operates on multi-modal features.

Decision Making Disease Prediction +2

GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images

no code implementations19 Mar 2021 Aadhithya Sankar, Matthias Keicher, Rami Eisawy, Abhijeet Parida, Franz Pfister, Seong Tae Kim, Nassir Navab

Disentangled representations can be useful in many downstream tasks, help to make deep learning models more interpretable, and allow for control over features of synthetically generated images that can be useful in training other models that require a large number of labelled or unlabelled data.


Longitudinal Quantitative Assessment of COVID-19 Infection Progression from Chest CTs

1 code implementation12 Mar 2021 Seong Tae Kim, Leili Goli, Magdalini Paschali, Ashkan Khakzar, Matthias Keicher, Tobias Czempiel, Egon Burian, Rickmer Braren, Nassir Navab, Thomas Wendler

Chest computed tomography (CT) has played an essential diagnostic role in assessing patients with COVID-19 by showing disease-specific image features such as ground-glass opacity and consolidation.

Computed Tomography (CT) COVID-19 Image Segmentation +1

OperA: Attention-Regularized Transformers for Surgical Phase Recognition

no code implementations5 Mar 2021 Tobias Czempiel, Magdalini Paschali, Daniel Ostler, Seong Tae Kim, Benjamin Busam, Nassir Navab

In this paper we introduce OperA, a transformer-based model that accurately predicts surgical phases from long video sequences.

Semi-Supervised Federated Peer Learning for Skin Lesion Classification

1 code implementation5 Mar 2021 Tariq Bdair, Nassir Navab, Shadi Albarqouni

With few annotated data, FedPerl is on par with a state-of-the-art method in skin lesion classification in the standard setup while outperforming SSFLs and the baselines by 1. 8% and 15. 8%, respectively.

Classification Federated Learning +4

RA-GCN: Graph Convolutional Network for Disease Prediction Problems with Imbalanced Data

no code implementations27 Feb 2021 Mahsa Ghorbani, Anees Kazi, Mahdieh Soleymani Baghshah, Hamid R. Rabiee, Nassir Navab

This is accomplished by associating a graph-based neural network to each class, which is responsible for weighting the class samples and changing the importance of each sample for the classifier.

Disease Prediction Node Classification

Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation

1 code implementation20 Dec 2020 Haowen Deng, Mai Bui, Nassir Navab, Leonidas Guibas, Slobodan Ilic, Tolga Birdal

For the former we contributed our own dataset composed of five indoor scenes where it is unavoidable to capture images corresponding to views that are hard to uniquely identify.

Camera Relocalization Pose Estimation

An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation

1 code implementation6 Dec 2020 Roger D. Soberanis-Mukul, Nassir Navab, Shadi Albarqouni

In this context, we proposed a segmentation refinement method based on uncertainty analysis and graph convolutional networks.

Graph Learning

Rethinking Positive Aggregation and Propagation of Gradients in Gradient-based Saliency Methods

no code implementations1 Dec 2020 Ashkan Khakzar, Soroosh Baselizadeh, Nassir Navab

In this work, we empirically show that two approaches for handling the gradient information, namely positive aggregation, and positive propagation, break these methods.

Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration

no code implementations14 Nov 2020 Matthias Grimm, Javier Esteban, Mathias Unberath, Nassir Navab

First, a neural network is trained once to detect a set of anatomical landmarks on simulated X-rays.

Self-Supervised Out-of-Distribution Detection in Brain CT Scans

no code implementations10 Nov 2020 Abinav Ravi Venkatakrishnan, Seong Tae Kim, Rami Eisawy, Franz Pfister, Nassir Navab

To address these issues, recently, unsupervised deep anomaly detection methods that train the model on large-sized normal scans and detect abnormal scans by calculating reconstruction error have been reported.

Anomaly Detection Out-of-Distribution Detection +1

Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds

no code implementations28 Oct 2020 Stefano Gasperini, Mohammad-Ali Nikouei Mahani, Alvaro Marcos-Ramiro, Nassir Navab, Federico Tombari

Panoptic segmentation has recently unified semantic and instance segmentation, previously addressed separately, thus taking a step further towards creating more comprehensive and efficient perception systems.

Instance Segmentation Panoptic Segmentation

SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion

2 code implementations26 Oct 2020 Shun-Cheng Wu, Keisuke Tateno, Nassir Navab, Federico Tombari

We propose a framework that ameliorates this issue by performing scene reconstruction and semantic scene completion jointly in an incremental and real-time manner, based on an input sequence of depth maps.

3D Semantic Scene Completion

RGB-D SLAM with Structural Regularities

1 code implementation15 Oct 2020 Yanyan Li, Raza Yunus, Nikolas Brasch, Nassir Navab, Federico Tombari

This work proposes a RGB-D SLAM system specifically designed for structured environments and aimed at improved tracking and mapping accuracy by relying on geometric features that are extracted from the surrounding.


I Like to Move It: 6D Pose Estimation as an Action Decision Process

no code implementations26 Sep 2020 Benjamin Busam, Hyun Jun Jung, Nassir Navab

We change this paradigm and reformulate the problem as an action decision process where an initial pose is updated in incremental discrete steps that sequentially move a virtual 3D rendering towards the correct solution.

6D Pose Estimation Pose Prediction +1

Polyp-artifact relationship analysis using graph inductive learned representations

no code implementations15 Sep 2020 Roger D. Soberanis-Mukul, Shadi Albarqouni, Nassir Navab

In inference, we use this classifier to analyze a second graph, generated from artifact and polyp predictions given by region proposal networks.

Graph Representation Learning Object Localization +1

Inverse Distance Aggregation for Federated Learning with Non-IID Data

no code implementations17 Aug 2020 Yousef Yeganeh, Azade Farshad, Nassir Navab, Shadi Albarqouni

Federated learning (FL) has been a promising approach in the field of medical imaging in recent years.

Federated Learning

A Learning-based Method for Online Adjustment of C-arm Cone-Beam CT Source Trajectories for Artifact Avoidance

no code implementations14 Aug 2020 Mareike Thies, Jan-Nico Zäch, Cong Gao, Russell Taylor, Nassir Navab, Andreas Maier, Mathias Unberath

We propose to adjust the C-arm CBCT source trajectory during the scan to optimize reconstruction quality with respect to a certain task, i. e. verification of screw placement.

Anatomy Tomographic Reconstructions

Continual Class Incremental Learning for CT Thoracic Segmentation

no code implementations12 Aug 2020 Abdelrahman Elskhawy, Aneta Lisowska, Matthias Keicher, Josep Henry, Paul Thomson, Nassir Navab

In this work, we evaluate FT and LwF for class incremental learning in multi-organ segmentation using the publicly available AAPM dataset.

class-incremental learning Incremental Learning

Structure-SLAM: Low-Drift Monocular SLAM in Indoor Environments

1 code implementation5 Aug 2020 Yanyan Li, Nikolas Brasch, Yida Wang, Nassir Navab, Federico Tombari

In this paper a low-drift monocular SLAM method is proposed targeting indoor scenarios, where monocular SLAM often fails due to the lack of textured surfaces.


DynaMiTe: A Dynamic Local Motion Model with Temporal Constraints for Robust Real-Time Feature Matching

no code implementations31 Jul 2020 Patrick Ruhkamp, Ruiqi Gong, Nassir Navab, Benjamin Busam

Feature based visual odometry and SLAM methods require accurate and fast correspondence matching between consecutive image frames for precise camera pose estimation in real-time.

Pose Estimation Visual Odometry

Towards Unsupervised Learning for Instrument Segmentation in Robotic Surgery with Cycle-Consistent Adversarial Networks

no code implementations9 Jul 2020 Daniil Pakhomov, Wei Shen, Nassir Navab

Surgical tool segmentation in endoscopic images is an important problem: it is a crucial step towards full instrument pose estimation and it is used for integration of pre- and intra-operative images into the endoscopic view.

Image-to-Image Translation Pose Estimation +1

Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery

no code implementations8 Jul 2020 Daniil Pakhomov, Nassir Navab

To account for reduced accuracy of the discovered light-weight deep residual network and avoid adding any additional computational burden, we perform a differentiable search over dilation rates for residual units of our network.

Pose Estimation

Augment Yourself: Mixed Reality Self-Augmentation Using Optical See-through Head-mounted Displays and Physical Mirrors

no code implementations6 Jul 2020 Mathias Unberath, Kevin Yu, Roghayeh Barmaki, Alex Johnson, Nassir Navab

Consequently, most MR applications that are centered around the user, such as virtual dressing rooms or learning of body movements, cannot be realized with HMDs.

Anatomy Mixed Reality

Scale-Space Autoencoders for Unsupervised Anomaly Segmentation in Brain MRI

1 code implementation23 Jun 2020 Christoph Baur, Benedikt Wiestler, Shadi Albarqouni, Nassir Navab

Brain pathologies can vary greatly in size and shape, ranging from few pixels (i. e. MS lesions) to large, space-occupying tumors.

Efficient Ensemble Model Generation for Uncertainty Estimation with Bayesian Approximation in Segmentation

no code implementations21 May 2020 Hong Joo Lee, Seong Tae Kim, Hakmin Lee, Nassir Navab, Yong Man Ro

Experimental results show that the proposed method could provide useful uncertainty information by Bayesian approximation with the efficient ensemble model generation and improve the predictive performance.

Simultaneous imputation and disease classification in incomplete medical datasets using Multigraph Geometric Matrix Completion (MGMC)

1 code implementation14 May 2020 Gerome Vivar, Anees Kazi, Hendrik Burwinkel, Andreas Zwergal, Nassir Navab, Seyed-Ahmad Ahmadi

As a solution, we propose an end-to-end learning of imputation and disease prediction of incomplete medical datasets via Multigraph Geometric Matrix Completion (MGMC).

Classification Disease Prediction +3

Decision Support for Intoxication Prediction Using Graph Convolutional Networks

no code implementations2 May 2020 Hendrik Burwinkel, Matthias Keicher, David Bani-Harouni, Tobias Zellner, Florian Eyer, Nassir Navab, Seyed-Ahmad Ahmadi

Due to the time-sensitive nature of these cases, doctors are required to propose a correct diagnosis and intervention within a minimal time frame.

Explicit Domain Adaptation with Loosely Coupled Samples

no code implementations24 Apr 2020 Oliver Scheel, Loren Schwarz, Nassir Navab, Federico Tombari

In this work we propose a transfer learning framework, core of which is learning an explicit mapping between domains.

Autonomous Driving Domain Adaptation +2

Colon Shape Estimation Method for Colonoscope Tracking using Recurrent Neural Networks

no code implementations20 Apr 2020 Masahiro Oda, Holger R. Roth, Takayuki Kitasaka, Kazuhiro Furukawa, Ryoji Miyahara, Yoshiki Hirooka, Hidemi Goto, Nassir Navab, Kensaku MORI

We propose a colon deformation estimation method using RNN and obtain the colonoscope shape from electromagnetic sensors during its insertion into the colon.

Colonoscope tracking method based on shape estimation network

no code implementations20 Apr 2020 Masahiro Oda, Holger R. Roth, Takayuki Kitasaka, Kazuhiro Furukawa, Ryoji Miyahara, Yoshiki Hirooka, Nassir Navab, Kensaku MORI

We utilize the shape estimation network (SEN), which estimates deformed colon shape during colonoscope insertions.

6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference

2 code implementations ECCV 2020 Mai Bui, Tolga Birdal, Haowen Deng, Shadi Albarqouni, Leonidas Guibas, Slobodan Ilic, Nassir Navab

We present a multimodal camera relocalization framework that captures ambiguities and uncertainties with continuous mixture models defined on the manifold of camera poses.

Camera Relocalization

Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions

no code implementations CVPR 2020 Johanna Wald, Helisa Dhamo, Nassir Navab, Federico Tombari

In our work we focus on scene graphs, a data structure that organizes the entities of a scene in a graph, where objects are nodes and their relationships modeled as edges.

3D Semantic Segmentation Scene Understanding

Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study

1 code implementation7 Apr 2020 Christoph Baur, Stefan Denner, Benedikt Wiestler, Shadi Albarqouni, Nassir Navab

Deep unsupervised representation learning has recently led to new approaches in the field of Unsupervised Anomaly Detection (UAD) in brain MRI.

Anatomy Experimental Design +2

Semantic Image Manipulation Using Scene Graphs

1 code implementation CVPR 2020 Helisa Dhamo, Azade Farshad, Iro Laina, Nassir Navab, Gregory D. Hager, Federico Tombari, Christian Rupprecht

In our work, we address the novel problem of image manipulation from scene graphs, in which a user can edit images by merely applying changes in the nodes or edges of a semantic graph that is generated from the image.

Image Inpainting Image Manipulation +1

Confident Coreset for Active Learning in Medical Image Analysis

no code implementations5 Apr 2020 Seong Tae Kim, Farrukh Mushtaq, Nassir Navab

Active learning is one of the solutions to this problem where an active learner is designed to indicate which samples need to be annotated to effectively train a target model.

Active Learning

Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning

3 code implementations30 Mar 2020 Hannes Hase, Mohammad Farid Azampour, Maria Tirindelli, Magdalini Paschali, Walter Simson, Emad Fatemizadeh, Nassir Navab

In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input.


Latent-Graph Learning for Disease Prediction

no code implementations27 Mar 2020 Luca Cosmo, Anees Kazi, Seyed-Ahmad Ahmadi, Nassir Navab, Michael Bronstein

Recently, Graph Convolutional Networks (GCNs) have proven to be a powerful machine learning tool for Computer-Aided Diagnosis (CADx) and disease prediction.

Disease Prediction General Classification +1

TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks

2 code implementations24 Mar 2020 Tobias Czempiel, Magdalini Paschali, Matthias Keicher, Walter Simson, Hubertus Feussner, Seong Tae Kim, Nassir Navab

Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems.

Fairness by Learning Orthogonal Disentangled Representations

no code implementations ECCV 2020 Mhd Hasan Sarhan, Nassir Navab, Abouzar Eslami, Shadi Albarqouni

We explicitly enforce the meaningful representation to be agnostic to sensitive information by entropy maximization.

Disentanglement Fairness

From Perspective X-ray Imaging to Parallax-Robust Orthographic Stitching

no code implementations5 Mar 2020 Javad Fotouhi, Xingtong Liu, Mehran Armand, Nassir Navab, Mathias Unberath

Stitching images acquired under perspective projective geometry is a relevant topic in computer vision with multiple applications ranging from smartphone panoramas to the construction of digital maps.

Anatomy Image Stitching

Spatiotemporal-Aware Augmented Reality: Redefining HCI in Image-Guided Therapy

no code implementations4 Mar 2020 Javad Fotouhi, Arian Mehrfard, Tianyu Song, Alex Johnson, Greg Osgood, Mathias Unberath, Mehran Armand, Nassir Navab

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies.

Anatomy Management

On the Effectiveness of Virtual Reality-based Training for Robotic Setup

no code implementations3 Mar 2020 Arian Mehrfard, Javad Fotouhi, Tess Forster, Giacomo Taylor, Danyal Fer, Deborah Nagle, Nassir Navab, Bernhard Fuerst

We trained 30 participants on how to set up a robotic arm in an environment mimicking clinical setup.


Force-Ultrasound Fusion: Bringing Spine Robotic-US to the Next "Level"

1 code implementation26 Feb 2020 Maria Tirindelli, Maria Victorova, Javier Esteban, Seong Tae Kim, David Navarro-Alarcon, Yong Ping Zheng, Nassir Navab

Processed force and ultrasound data are fused using a 1D Convolutional Network to compute the location of the vertebral levels.

Differentiable Graph Module (DGM) for Graph Convolutional Networks

1 code implementation11 Feb 2020 Anees Kazi, Luca Cosmo, Seyed-Ahmad Ahmadi, Nassir Navab, Michael Bronstein

We provide an extensive evaluation of applications from the domains of healthcare (disease prediction), brain imaging (age prediction), computer graphics (3D point cloud segmentation), and computer vision (zero-shot learning).

Disease Prediction Point Cloud Segmentation +1

Understanding the effects of artifacts on automated polyp detection and incorporating that knowledge via learning without forgetting

1 code implementation7 Feb 2020 Maxime Kayser, Roger D. Soberanis-Mukul, Anna-Maria Zvereva, Peter Klare, Nassir Navab, Shadi Albarqouni

We then investigated different strategies, such as a learning without forgetting framework, to leverage artifact knowledge to improve automated polyp detection.

Object Localization

A Comparative Analysis of Virtual Reality Head-Mounted Display Systems

no code implementations5 Dec 2019 Arian Mehrfard, Javad Fotouhi, Giacomo Taylor, Tess Forster, Nassir Navab, Bernhard Fuerst

With recent advances of Virtual Reality (VR) technology, the deployment of such will dramatically increase in non-entertainment environments, such as professional education and training, manufacturing, service, or low frequency/high risk scenarios.

Improving Feature Attribution through Input-specific Network Pruning

no code implementations25 Nov 2019 Ashkan Khakzar, Soroosh Baselizadeh, Saurabh Khanduja, Christian Rupprecht, Seong Tae Kim, Nassir Navab

Attributing the output of a neural network to the contribution of given input elements is a way of shedding light on the black-box nature of neural networks.

Network Pruning

Radar Emitter Classification with Attribute-specific Recurrent Neural Networks

no code implementations18 Nov 2019 Paolo Notaro, Magdalini Paschali, Carsten Hopke, David Wittmann, Nassir Navab

Radar pulse streams exhibit increasingly complex temporal patterns and can no longer rely on a purely value-based analysis of the pulse attributes for the purpose of emitter classification.

Classification General Classification

Signal Clustering with Class-independent Segmentation

no code implementations18 Nov 2019 Stefano Gasperini, Magdalini Paschali, Carsten Hopke, David Wittmann, Nassir Navab

Radar signals have been dramatically increasing in complexity, limiting the source separation ability of traditional approaches.

Semantic Segmentation

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

no code implementations20 Oct 2019 Tom Vercauteren, Mathias Unberath, Nicolas Padoy, Nassir Navab

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice.

Decision Making

Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories

no code implementations19 Sep 2019 Jan-Nico Zaech, Cong Gao, Bastian Bier, Russell Taylor, Andreas Maier, Nassir Navab, Mathias Unberath

Metal artifacts in computed tomography (CT) arise from a mismatch between physics of image formation and idealized assumptions during tomographic reconstruction.

Computed Tomography (CT)

Learn to Estimate Labels Uncertainty for Quality Assurance

no code implementations17 Sep 2019 Agnieszka Tomczack, Nassir Navab, Shadi Albarqouni

Deep Learning sets the state-of-the-art in many challenging tasks showing outstanding performance in a broad range of applications.

Bayesian Inference

Learn to Segment Organs with a Few Bounding Boxes

no code implementations17 Sep 2019 Abhijeet Parida, Arianne Tran, Nassir Navab, Shadi Albarqouni

Semantic segmentation is an import task in the medical field to identify the exact extent and orientation of significant structures like organs and pathology.

Semantic Segmentation

ForkNet: Multi-branch Volumetric Semantic Completion from a Single Depth Image

no code implementations ICCV 2019 Yida Wang, David Joseph Tan, Nassir Navab, Federico Tombari

We propose a novel model for 3D semantic completion from a single depth image, based on a single encoder and three separate generators used to reconstruct different geometric and semantic representations of the original and completed scene, all sharing the same latent space.

Ranked #7 on 3D Semantic Scene Completion on NYUv2 (using extra training data)

3D Semantic Scene Completion

Object-Driven Multi-Layer Scene Decomposition From a Single Image

no code implementations ICCV 2019 Helisa Dhamo, Nassir Navab, Federico Tombari

Our approach aims at building up a Layered Depth Image (LDI) from a single RGB input, which is an efficient representation that arranges the scene in layers, including originally occluded regions.

RIO: 3D Object Instance Re-Localization in Changing Indoor Environments

no code implementations ICCV 2019 Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, Matthias Nießner

In this work, we introduce the task of 3D object instance re-localization (RIO): given one or multiple objects in an RGB-D scan, we want to estimate their corresponding 6DoF poses in another 3D scan of the same environment taken at a later point in time.

Scene Understanding

Few-Shot Meta-Denoising

no code implementations31 Jul 2019 Leslie Casas, Attila Klimmek, Gustavo Carneiro, Nassir Navab, Vasileios Belagiannis

A solution to mitigate the small training set issue is to pre-train a denoising model with small training sets containing pairs of clean and synthesized noisy signals, produced from empirical noise priors, and fine-tune on the available small training set.

Denoising Few-Shot Learning +1

Reflective-AR Display: An Interaction Methodology for Virtual-Real Alignment in Medical Robotics

no code implementations23 Jul 2019 Javad Fotouhi, Tianyu Song, Arian Mehrfard, Giacomo Taylor, Qiaochu Wang, Fengfang Xian, Alejandro Martin-Gomez, Bernhard Fuerst, Mehran Armand, Mathias Unberath, Nassir Navab

To overcome this challenge, we introduce a novel registration concept for intuitive alignment of AR content to its physical counterpart by providing a multi-view AR experience via reflective-AR displays that simultaneously show the augmentations from multiple viewpoints.

Image to Images Translation for Multi-Task Organ Segmentation and Bone Suppression in Chest X-Ray Radiography

1 code implementation24 Jun 2019 Mohammad Eslami, Solale Tabarestani, Shadi Albarqouni, Ehsan Adeli, Nassir Navab, Malek Adjouadi

Chest X-ray radiography is one of the earliest medical imaging technologies and remains one of the most widely-used for diagnosis, screening, and treatment follow up of diseases related to lungs and heart.

Decision Making Translation

`Project & Excite' Modules for Segmentation of Volumetric Medical Scans

2 code implementations11 Jun 2019 Anne-Marie Rickmann, Abhijit Guha Roy, Ignacio Sarasua, Nassir Navab, Christian Wachinger

Fully Convolutional Neural Networks (F-CNNs) achieve state-of-the-art performance for image segmentation in medical imaging.

Brain Segmentation Semantic Segmentation

Uncertainty-based graph convolutional networks for organ segmentation refinement

1 code implementation MIDL 2019 Roger D. Soberanis-Mukul, Nassir Navab, Shadi Albarqouni

In this context, we proposed a segmentation refinement method based on uncertainty analysis and graph convolutional networks.

Graph Learning

Perceptual Embedding Consistency for Seamless Reconstruction of Tilewise Style Transfer

no code implementations3 Jun 2019 Amal Lahiani, Nassir Navab, Shadi Albarqouni, Eldad Klaiman

Recent work has shown Generative Adversarial Networks(GANs) can be used to create realistic images of virtually stained slide images in digital pathology with clinically validated interpretability.

Style Transfer

BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning

no code implementations16 May 2019 Abhijit Guha Roy, Shayan Siddiqui, Sebastian Pölsterl, Nassir Navab, Christian Wachinger

A disadvantage of FL is the dependence on a central server, which requires all clients to agree on one trusted central body, and whose failure would disrupt the training process of all clients.

Brain Segmentation Federated Learning

Learning Interpretable Features via Adversarially Robust Optimization

no code implementations9 May 2019 Ashkan Khakzar, Shadi Albarqouni, Nassir Navab

In this work, we propose a method for improving the feature interpretability of neural network classifiers.

Decision Making

Adaptive Image-Feature Learning for Disease Classification Using Inductive Graph Networks

no code implementations8 May 2019 Hendrik Burwinkel, Anees Kazi, Gerome Vivar, Shadi Albarqouni, Guillaume Zahnd, Nassir Navab, Seyed-Ahmad Ahmadi

We propose a new network architecture that exploits an inductive end-to-end learning approach for disease classification, where filters from both the CNN and the graph are trained jointly.

Classification General Classification

Multi-modal Graph Fusion for Inductive Disease Classification in Incomplete Datasets

no code implementations8 May 2019 Gerome Vivar, Hendrik Burwinkel, Anees Kazi, Andreas Zwergal, Nassir Navab, Seyed-Ahmad Ahmadi

Recently, several works proposed geometric deep learning approaches to solve disease classification, by modeling patients as nodes in a graph, along with graph signal processing of multi-modal features.

Classification Decision Making +1

Fully Automatic Segmentation of 3D Brain Ultrasound: Learning from Coarse Annotations

no code implementations18 Apr 2019 Julia Rackerseder, Rüdiger Göbl, Nassir Navab, Christoph Hennersperger

Trained on the dataset alone, we report a Dice and Jaccard coefficient of $0. 45 \pm 0. 09$ and $0. 30 \pm 0. 07$ respectively, as well as an average distance of $0. 78 \pm 0. 36~mm$.

Multi-scale Microaneurysms Segmentation Using Embedding Triplet Loss

no code implementations18 Apr 2019 Mhd Hasan Sarhan, Shadi Albarqouni, Mehmet Yigitsoy, Nassir Navab, Abouzar Eslami

To enhance the discriminative power of the classification model, we incorporate triplet embedding loss with a selective sampling routine.

Diabetic Retinopathy Detection General Classification

Collaboration Analysis Using Deep Learning

no code implementations17 Apr 2019 Zhang Guo, Kevin Yu, Rebecca Pearlman, Nassir Navab, Roghayeh Barmaki

The analysis of the collaborative learning process is one of the growing fields of education research, which has many different analytic solutions.

Anatomy Object Recognition

Learning Interpretable Disentangled Representations using Adversarial VAEs

no code implementations17 Apr 2019 Mhd Hasan Sarhan, Abouzar Eslami, Nassir Navab, Shadi Albarqouni

Learning Interpretable representation in medical applications is becoming essential for adopting data-driven models into clinical practice.

Disentanglement General Classification

Weakly-Supervised White and Grey Matter Segmentation in 3D Brain Ultrasound

no code implementations10 Apr 2019 Beatrice Demiray, Julia Rackerseder, Stevica Bozhinoski, Nassir Navab

We implement label transfer from MRI to US, which is prone to a residual but inevitable registration error.

Transfer Learning

End-to-End Learning-Based Ultrasound Reconstruction

no code implementations9 Apr 2019 Walter Simson, Rüdiger Göbl, Magdalini Paschali, Markus Krönke, Klemens Scheidhauer, Wolfgang Weber, Nassir Navab

The proposed method displays both promising image reconstruction quality and acquisition frequency when integrated for live ultrasound scanning.

Image Reconstruction

Adversarial Networks for Camera Pose Regression and Refinement

no code implementations15 Mar 2019 Mai Bui, Christoph Baur, Nassir Navab, Slobodan Ilic, Shadi Albarqouni

Despite recent advances on the topic of direct camera pose regression using neural networks, accurately estimating the camera pose of a single RGB image still remains a challenging task.

Pose Estimation