Search Results for author: Nassir Navab

Found 346 papers, 108 papers with code

RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance

1 code implementation30 Nov 2023 Chantal Pellegrini, Ege Özsoy, Benjamin Busam, Nassir Navab, Matthias Keicher

Conversational AI tools that can generate and discuss clinically correct radiology reports for a given medical image have the potential to transform radiology.

Language Modelling Large Language Model

Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks

no code implementations20 Nov 2023 Mayar Lotfy, Anna Alperovich, Tommaso Giannantonio, Bjorn Barz, Xiaohan Zhang, Felix Holm, Nassir Navab, Felix Boehm, Carolin Schwamborn, Thomas K. Hoffmann, Patrick J. Schuler

Despite the limited dataset, the GNN-based model significantly outperforms context-agnostic approaches, accurately distinguishing between healthy and tumor tissues, even in images from previously unseen patients.

Tumor Segmentation

SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation

no code implementations18 Nov 2023 Yamei Chen, Yan Di, Guangyao Zhai, Fabian Manhardt, Chenyangguang Zhang, Ruida Zhang, Federico Tombari, Nassir Navab, Benjamin Busam

These geometric features are then point-aligned with DINOv2 features to establish a consistent object representation under SE(3) transformations, facilitating the mapping from camera space to the pre-defined canonical space, thus further enhancing pose estimation.

Pose Estimation

EyeLS: Shadow-Guided Instrument Landing System for Intraocular Target Approaching in Robotic Eye Surgery

no code implementations15 Nov 2023 Junjie Yang, Zhihao Zhao, Siyuan Shen, Daniel Zapp, Mathias Maier, Kai Huang, Nassir Navab, M. Ali Nasseri

Robotic ophthalmic surgery is an emerging technology to facilitate high-precision interventions such as retina penetration in subretinal injection and removal of floating tissues in retinal detachment depending on the input imaging modalities such as microscopy and intraoperative OCT (iOCT).

VoxNeRF: Bridging Voxel Representation and Neural Radiance Fields for Enhanced Indoor View Synthesis

no code implementations9 Nov 2023 Sen Wang, Wei zhang, Stefano Gasperini, Shun-Cheng Wu, Nassir Navab

Creating high-quality view synthesis is essential for immersive applications but continues to be problematic, particularly in indoor environments and for real-time deployment.

PRISM: Progressive Restoration for Scene Graph-based Image Manipulation

no code implementations3 Nov 2023 Pavel Jahoda, Azade Farshad, Yousef Yeganeh, Ehsan Adeli, Nassir Navab

We take advantage of the outer part of the masked area as they have a direct correlation with the context of the scene.

Denoising Descriptive +2

AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers

no code implementations25 Sep 2023 Alex Ranne, Yordanka Velikova, Nassir Navab, Ferdinando Rodriguez y Baena

To date, endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature.

Dynamic Scene Graph Representation for Surgical Video

no code implementations25 Sep 2023 Felix Holm, Ghazal Ghazaei, Tobias Czempiel, Ege Özsoy, Stefan Saur, Nassir Navab

Surgical videos captured from microscopic or endoscopic imaging devices are rich but complex sources of information, depicting different tools and anatomical structures utilized during an extended amount of time.

SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs

no code implementations21 Sep 2023 Guangyao Zhai, Xiaoni Cai, Dianye Huang, Yan Di, Fabian Manhardt, Federico Tombari, Nassir Navab, Benjamin Busam

In this paper, we present SG-Bot, a novel rearrangement framework that utilizes a coarse-to-fine scheme with a scene graph as the scene representation.

RIDE: Self-Supervised Learning of Rotation-Equivariant Keypoint Detection and Invariant Description for Endoscopy

no code implementations18 Sep 2023 Mert Asim Karaoglu, Viktoria Markova, Nassir Navab, Benjamin Busam, Alexander Ladikos

While most classical methods achieve rotation-equivariant detection and invariant description by design, many learning-based approaches learn to be robust only up to a certain degree.

Keypoint Detection Self-Supervised Learning

Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstruction

no code implementations ICCV 2023 Zhiying Leng, Shun-Cheng Wu, Mahdi Saleh, Antonio Montanaro, Hao Yu, Yin Wang, Nassir Navab, Xiaohui Liang, Federico Tombari

In this work, we propose the first precise hand-object reconstruction method in hyperbolic space, namely Dynamic Hyperbolic Attention Network (DHANet), which leverages intrinsic properties of hyperbolic space to learn representative features.

Object Reconstruction

BigFUSE: Global Context-Aware Image Fusion in Dual-View Light-Sheet Fluorescence Microscopy with Image Formation Prior

no code implementations5 Sep 2023 Yu Liu, Gesine Muller, Nassir Navab, Carsten Marr, Jan Huisken, Tingying Peng

Light-sheet fluorescence microscopy (LSFM), a planar illumination technique that enables high-resolution imaging of samples, experiences defocused image quality caused by light scattering when photons propagate through thick tissues.

On the Localization of Ultrasound Image Slices within Point Distribution Models

1 code implementation1 Sep 2023 Lennart Bastian, Vincent Bürgin, Ha Young Kim, Alexander Baumann, Benjamin Busam, Mahdi Saleh, Nassir Navab

We demonstrate that our multi-modal registration framework can localize images on the 3D surface topology of a patient-specific organ and the mean shape of an SSM.

3D Reconstruction 3D Shape Representation +2

3D Adversarial Augmentations for Robust Out-of-Domain Predictions

no code implementations29 Aug 2023 Alexander Lehner, Stefano Gasperini, Alvaro Marcos-Ramiro, Michael Schmidt, Nassir Navab, Benjamin Busam, Federico Tombari

We conduct extensive experiments across a variety of scenarios on data from KITTI, Waymo, and CrashD for 3D object detection, and on data from SemanticKITTI, Waymo, and nuScenes for 3D semantic segmentation.

3D Object Detection 3D Semantic Segmentation +2

A Continual Learning Approach for Cross-Domain White Blood Cell Classification

no code implementations24 Aug 2023 Ario Sadafi, Raheleh Salehi, Armin Gruber, Sayedali Shetab Boushehri, Pascal Giehr, Nassir Navab, Carsten Marr

Here, we propose a rehearsal-based continual learning approach for class incremental and domain incremental scenarios in white blood cell classification.

Classification Continual Learning

A Study of Age and Sex Bias in Multiple Instance Learning based Classification of Acute Myeloid Leukemia Subtypes

no code implementations24 Aug 2023 Ario Sadafi, Matthias Hehr, Nassir Navab, Carsten Marr

To that end, we train multiple MIL models using different levels of sex imbalance in the training set and excluding certain age groups.

Classification Decision Making +1

Multi-Modal Dataset Acquisition for Photometrically Challenging Object

no code implementations21 Aug 2023 HyunJun Jung, Patrick Ruhkamp, Nassir Navab, Benjamin Busam

This paper addresses the limitations of current datasets for 3D vision tasks in terms of accuracy, size, realism, and suitable imaging modalities for photometrically challenging objects.

Polarimetric Information for Multi-Modal 6D Pose Estimation of Photometrically Challenging Objects with Limited Data

no code implementations21 Aug 2023 Patrick Ruhkamp, Daoyi Gao, HyunJun Jung, Nassir Navab, Benjamin Busam

6D pose estimation pipelines that rely on RGB-only or RGB-D data show limitations for photometrically challenging objects with e. g. textureless surfaces, reflections or transparency.

6D Pose Estimation

Robust Monocular Depth Estimation under Challenging Conditions

no code implementations ICCV 2023 Stefano Gasperini, Nils Morbitzer, HyunJun Jung, Nassir Navab, Federico Tombari

While state-of-the-art monocular depth estimation approaches achieve impressive results in ideal settings, they are highly unreliable under challenging illumination and weather conditions, such as at nighttime or in the presence of rain.

Monocular Depth Estimation valid

DISBELIEVE: Distance Between Client Models is Very Essential for Effective Local Model Poisoning Attacks

no code implementations14 Aug 2023 Indu Joshi, Priyank Upadhya, Gaurav Kumar Nayak, Peter Schüffler, Nassir Navab

Leveraging this, we introduce DISBELIEVE, a local model poisoning attack that creates malicious parameters or gradients such that their distance to benign clients' parameters or gradients is low respectively but at the same time their adverse effect on the global model's performance is high.

Federated Learning Model Poisoning +1

WarpEM: Dynamic Time Warping for Accurate Catheter Registration in EM-guided Procedures

no code implementations7 Aug 2023 Ardit Ramadani, Peter Ewert, Heribert Schunkert, Nassir Navab

Accurate catheter tracking is crucial during minimally invasive endovascular procedures (MIEP), and electromagnetic (EM) tracking is a widely used technology that serves this purpose.

Dynamic Time Warping Medical Procedure

DefCor-Net: Physics-Aware Ultrasound Deformation Correction

1 code implementation7 Aug 2023 Zhongliang Jiang, Yue Zhou, Dongliang Cao, Nassir Navab

The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis.


LOTUS: Learning to Optimize Task-based US representations

1 code implementation29 Jul 2023 Yordanka Velikova, Mohammad Farid Azampour, Walter Simson, Vanessa Gonzalez Duque, Nassir Navab

Anatomical segmentation of organs in ultrasound images is essential to many clinical applications, particularly for diagnosis and monitoring.

Image Generation Segmentation

Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures

1 code implementation27 Jul 2023 Kun Yuan, Vinkle Srivastav, Tong Yu, Joel Lavanchy, Pietro Mascagni, Nassir Navab, Nicolas Padoy

SurgVLP constructs a new contrastive learning objective to align video clip embeddings with the corresponding multiple text embeddings by bringing them together within a joint latent space.

Automatic Speech Recognition Contrastive Learning +6

DisguisOR: Holistic Face Anonymization for the Operating Room

1 code implementation26 Jul 2023 Lennart Bastian, Tony Danjun Wang, Tobias Czempiel, Benjamin Busam, Nassir Navab

Methods: RGB and depth images from multiple cameras are fused into a 3D point cloud representation of the scene.

Face Anonymization

DISA: DIfferentiable Similarity Approximation for Universal Multimodal Registration

1 code implementation19 Jul 2023 Matteo Ronchetti, Wolfgang Wein, Nassir Navab, Oliver Zettinig, Raphael Prevost

Our method is several orders of magnitude faster than local patch-based metrics and can be directly applied in clinical settings by replacing the similarity measure with the proposed one.

Anatomy Image Registration

Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting

1 code implementation11 Jul 2023 Chantal Pellegrini, Matthias Keicher, Ege Özsoy, Nassir Navab

However, there is limited research on automating structured reporting, and no public benchmark is available for evaluating and comparing different methods.

Medical Visual Question Answering Question Answering +2

Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations

1 code implementation7 Jul 2023 Zhongliang Jiang, Yuan Bi, Mingchuan Zhou, Ying Hu, Michael Burke, Nassir Navab

The results demonstrated that the proposed advanced framework can robustly work on a variety of seen and unseen phantoms as well as in-vivo human carotid data.


Thoracic Cartilage Ultrasound-CT Registration using Dense Skeleton Graph

1 code implementation7 Jul 2023 Zhongliang Jiang, Chenyang Li, Xuesong Li, Nassir Navab

To address this challenge, a graph-based non-rigid registration is proposed to enable transferring planned paths from the atlas to the current setup by explicitly considering subcutaneous bone surface features instead of the skin surface.

Template Matching

Motion Magnification in Robotic Sonography: Enabling Pulsation-Aware Artery Segmentation

1 code implementation7 Jul 2023 Dianye Huang, Yuan Bi, Nassir Navab, Zhongliang Jiang

To validate the proposed robotic US system for imaging arteries, experiments are carried out on volunteers' carotid and radial arteries.

Motion Magnification Segmentation

AutoPaint: A Self-Inpainting Method for Unsupervised Anomaly Detection

no code implementations21 May 2023 Mehdi Astaraki, Francesca De Benetti, Yousef Yeganeh, Iuliana Toma-Dasu, Örjan Smedby, Chunliang Wang, Nassir Navab, Thomas Wendler

This work intends to, first, propose a robust inpainting model to learn the details of healthy anatomies and reconstruct high-resolution images by preserving anatomical constraints.

Unsupervised Anomaly Detection

Self-Supervised Learning for Physiologically-Based Pharmacokinetic Modeling in Dynamic PET

no code implementations17 May 2023 Francesca De Benetti, Walter Simson, Magdalini Paschali, Hasan Sari, Axel Romiger, Kuangyu Shi, Nassir Navab, Thomas Wendler

Dynamic positron emission tomography imaging (dPET) provides temporally resolved images of a tracer enabling a quantitative measure of physiological processes.

Self-Supervised Learning

DopUS-Net: Quality-Aware Robotic Ultrasound Imaging based on Doppler Signal

1 code implementation15 May 2023 Zhongliang Jiang, Felix Duelmer, Nassir Navab

The experimental results demonstrate that the proposed approach with the re-identification process can significantly improve the accuracy and robustness of the segmentation results (dice score: from 0:54 to 0:86; intersection over union: from 0:47 to 0:78).

Image Segmentation Region Proposal +2

Skeleton Graph-based Ultrasound-CT Non-rigid Registration

no code implementations14 May 2023 Zhongliang Jiang, Xuesong Li, Chenyu Zhang, Yuan Bi, Walter Stechele, Nassir Navab

Autonomous ultrasound (US) scanning has attracted increased attention, and it has been seen as a potential solution to overcome the limitations of conventional US examinations, such as inter-operator variations.

Next-generation Surgical Navigation: Multi-view Marker-less 6DoF Pose Estimation of Surgical Instruments

no code implementations5 May 2023 Jonas Hein, Nicola Cavalcanti, Daniel Suter, Lukas Zingg, Fabio Carrillo, Mazda Farshad, Marc Pollefeys, Nassir Navab, Philipp Fürnstahl

A particular focus in computer-assisted surgery is to replace marker-based tracking systems for instrument localization with pure image-based 6DoF pose estimation.

Pose Estimation

Incremental 3D Semantic Scene Graph Prediction from RGB Sequences

no code implementations CVPR 2023 Shun-Cheng Wu, Keisuke Tateno, Nassir Navab, Federico Tombari

Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network.

SCOPE: Structural Continuity Preservation for Medical Image Segmentation

no code implementations28 Apr 2023 Yousef Yeganeh, Azade Farshad, Goktug Guevercin, Amr Abu-zer, Rui Xiao, Yongjian Tang, Ehsan Adeli, Nassir Navab

Although the preservation of shape continuity and physiological anatomy is a natural assumption in the segmentation of medical images, it is often neglected by deep learning methods that mostly aim for the statistical modeling of input data as pixels rather than interconnected structures.

Anatomy Image Segmentation +3

DIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation

no code implementations28 Apr 2023 Yousef Yeganeh, Azade Farshad, Peter Weinberger, Seyed-Ahmad Ahmadi, Ehsan Adeli, Nassir Navab

Although purely transformer-based architectures showed promising performance in many computer vision tasks, many hybrid models consisting of CNN and transformer blocks are introduced to fit more specialized tasks.

Image Segmentation Medical Image Segmentation +1

SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis

no code implementations28 Apr 2023 Azade Farshad, Yousef Yeganeh, Yu Chi, Chengzhi Shen, Björn Ommer, Nassir Navab

To address this limitation, we propose a novel guidance approach for the sampling process in the diffusion model that leverages bounding box and segmentation map information at inference time without additional training data.

Image Generation from Scene Graphs Segmentation

S3M: Scalable Statistical Shape Modeling through Unsupervised Correspondences

1 code implementation15 Apr 2023 Lennart Bastian, Alexander Baumann, Emily Hoppe, Vincent Bürgin, Ha Young Kim, Mahdi Saleh, Benjamin Busam, Nassir Navab

Statistical shape models (SSMs) are an established way to represent the anatomy of a population with various clinically relevant applications.


Prior-RadGraphFormer: A Prior-Knowledge-Enhanced Transformer for Generating Radiology Graphs from X-Rays

1 code implementation24 Mar 2023 Yiheng Xiong, Jingsong Liu, Kamilia Zaripova, Sahand Sharifzadeh, Matthias Keicher, Nassir Navab

The extraction of structured clinical information from free-text radiology reports in the form of radiology graphs has been demonstrated to be a valuable approach for evaluating the clinical correctness of report-generation methods.

Decision Making Multi-Label Classification +1

LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Rooms

no code implementations23 Mar 2023 Ege Özsoy, Tobias Czempiel, Felix Holm, Chantal Pellegrini, Nassir Navab

The holistic representation of surgical scenes as semantic scene graphs (SGG), where entities are represented as nodes and relations between them as edges, is a promising direction for fine-grained semantic OR understanding.

Scene Graph Generation

MI-SegNet: Mutual Information-Based US Segmentation for Unseen Domain Generalization

2 code implementations22 Mar 2023 Yuan Bi, Zhongliang Jiang, Ricarda Clarenbach, Reza Ghotbi, Angelos Karlas, Nassir Navab

We validate the generalizability of the proposed domain-independent segmentation approach on several datasets with varying parameters and machines.

Anatomy Disentanglement +5

Location-Free Scene Graph Generation

no code implementations20 Mar 2023 Ege Özsoy, Felix Holm, Tobias Czempiel, Nassir Navab, Benjamin Busam

Although using significantly fewer labels during training, we achieve 74. 12\% of the location-supervised SOTA performance on Visual Genome and even outperform the best method on 4D-OR.

Graph Generation Scene Graph Generation

Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs

no code implementations15 Mar 2023 Artem Savkin, Rachid Ellouze, Nassir Navab, Federico Tombari

Image synthesis driven by computer graphics achieved recently a remarkable realism, yet synthetic image data generated this way reveals a significant domain gap with respect to real-world data.

Autonomous Driving Image Generation +1

BEL: A Bag Embedding Loss for Transformer enhances Multiple Instance Whole Slide Image Classification

no code implementations2 Mar 2023 Daniel Sens, Ario Sadafi, Francesco Paolo Casale, Nassir Navab, Carsten Marr

Recent MIL approaches produce highly informative bag level representations by utilizing the transformer architecture's ability to model the dependencies between instances.

Image Classification Multiple Instance Learning +1

KST-Mixer: Kinematic Spatio-Temporal Data Mixer For Colon Shape Estimation

1 code implementation2 Feb 2023 Masahiro Oda, Kazuhiro Furukawa, Nassir Navab, Kensaku MORI

Kinematic data of a colonoscope and the colon, including positions and directions of their centerlines, are obtained using electromagnetic and depth sensors.

Lidar Upsampling with Sliced Wasserstein Distance

no code implementations31 Jan 2023 Artem Savkin, Yida Wang, Sebastian Wirkert, Nassir Navab, Federico Tombar

This in turn enables our method to employ a one-stage upsampling paradigm without the need for coarse and fine reconstruction.

Autonomous Driving Domain Adaptation

Ultra-NeRF: Neural Radiance Fields for Ultrasound Imaging

no code implementations25 Jan 2023 Magdalena Wysocki, Mohammad Farid Azampour, Christine Eilers, Benjamin Busam, Mehrdad Salehi, Nassir Navab

In our work, we discuss direction-dependent changes in the scene and show that a physics-inspired rendering improves the fidelity of US image synthesis.

Image Generation Neural Rendering

TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimation

no code implementations CVPR 2023 Hanzhi Chen, Fabian Manhardt, Nassir Navab, Benjamin Busam

In this paper, we introduce neural texture learning for 6D object pose estimation from synthetic data and a few unlabelled real images.

6D Pose Estimation using RGB

SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environments

1 code implementation22 Dec 2022 Evin Pınar Örnek, Aravindhan K Krishnan, Shreekant Gayaka, Cheng-Hao Kuo, Arnie Sen, Nassir Navab, Federico Tombari

We introduce a zero-shot split for Tabletop Objects Dataset (TOD-Z) to enable this study and present a method that uses annotated objects to learn the ``objectness'' of pixels and generalize to unseen object categories in cluttered indoor environments.

Instance Segmentation Semantic Segmentation

DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation

no code implementations10 Nov 2022 Azade Farshad, Yousef Yeganeh, Helisa Dhamo, Federico Tombari, Nassir Navab

Graph representation of objects and their relations in a scene, known as a scene graph, provides a precise and discernible interface to manipulate a scene by modifying the nodes or the edges in the graph.

Disentanglement Image Manipulation

Improved Techniques for the Conditional Generative Augmentation of Clinical Audio Data

no code implementations5 Nov 2022 Mane Margaryan, Matthias Seibold, Indu Joshi, Mazda Farshad, Philipp Fürnstahl, Nassir Navab

In contrast to previously proposed fully convolutional models, the proposed model implements residual Squeeze and Excitation modules in the generator architecture.

Data Augmentation

What can we learn about a generated image corrupting its latent representation?

no code implementations12 Oct 2022 Agnieszka Tomczak, Aarushi Gupta, Slobodan Ilic, Nassir Navab, Shadi Albarqouni

The purpose of this work is to investigate the hypothesis that we can predict image quality based on its latent representation in the GANs bottleneck.

Image-to-Image Translation Liver Segmentation

Segmenting Known Objects and Unseen Unknowns without Prior Knowledge

no code implementations ICCV 2023 Stefano Gasperini, Alvaro Marcos-Ramiro, Michael Schmidt, Nassir Navab, Benjamin Busam, Federico Tombari

By doing so, for the first time in panoptic segmentation with unknown objects, our U3HS is trained without unknown categories, reducing assumptions and leaving the settings as unconstrained as in real-life scenarios.

Panoptic Segmentation Scene Understanding +1

Towards Autonomous Atlas-based Ultrasound Acquisitions in Presence of Articulated Motion

1 code implementation10 Aug 2022 Zhongliang Jiang, Yuan Gao, Le Xie, Nassir Navab

Robotic ultrasound (US) imaging aims at overcoming some of the limitations of free-hand US examinations, e. g. difficulty in guaranteeing intra- and inter-operator repeatability.

Speckle2Speckle: Unsupervised Learning of Ultrasound Speckle Filtering Without Clean Data

1 code implementation31 Jul 2022 Rüdiger Göbl, Christoph Hennersperger, Nassir Navab

To enable this, we make use of realistic ultrasound simulation techniques that allow for instantiation of several independent speckle realizations that represent the exact same tissue, thus allowing for the application of image reconstruction techniques that work with pairs of differently corrupted data.

Image Reconstruction

DA$^2$ Dataset: Toward Dexterity-Aware Dual-Arm Grasping

no code implementations31 Jul 2022 Guangyao Zhai, Yu Zheng, Ziwei Xu, Xin Kong, Yong liu, Benjamin Busam, Yi Ren, Nassir Navab, Zhengyou Zhang

In this paper, we introduce DA$^2$, the first large-scale dual-arm dexterity-aware dataset for the generation of optimal bimanual grasping pairs for arbitrary large objects.

CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning

no code implementations31 Jul 2022 Mahdi Saleh, Yige Wang, Nassir Navab, Benjamin Busam, Federico Tombari

The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods while requiring significantly fewer computations.

Scene Segmentation Segmentation

Spotlight on nerves: Portable multispectral optoacoustic imaging of peripheral nerve vascularization and morphology

no code implementations28 Jul 2022 Dominik Jüstel, Hedwig Irl, Florian Hinterwimmer, Christoph Dehner, Walter Simson, Nassir Navab, Gerhard Schneider, Vasilis Ntziachristos

Various morphological and functional parameters of peripheral nerves and their vascular supply are indicative of pathological changes due to injury or disease.

Exploiting Diversity of Unlabeled Data for Label-Efficient Semi-Supervised Active Learning

no code implementations25 Jul 2022 Felix Buchert, Nassir Navab, Seong Tae Kim

By considering the consistency information with the diversity in the consistency-based embedding scheme, the proposed method could select more informative samples for labeling in the semi-supervised learning setting.

Active Learning Representation Learning

Unsupervised pre-training of graph transformers on patient population graphs

2 code implementations21 Jul 2022 Chantal Pellegrini, Nassir Navab, Anees Kazi

We find that our proposed pre-training methods help in modeling the data at a patient and population level and improve performance in different fine-tuning tasks on all datasets.

Language Modelling Masked Language Modeling +2

CACTUSS: Common Anatomical CT-US Space for US examinations

1 code implementation18 Jul 2022 Yordanka Velikova, Walter Simson, Mehrdad Salehi, Mohammad Farid Azampour, Philipp Paprottka, Nassir Navab

Abdominal aortic aneurysm (AAA) is a vascular disease in which a section of the aorta enlarges, weakening its walls and potentially rupturing the vessel.

Image-to-Image Translation Segmentation

Shape-Aware Masking for Inpainting in Medical Imaging

no code implementations12 Jul 2022 Yousef Yeganeh, Azade Farshad, Nassir Navab

Inpainting has recently been proposed as a successful deep learning technique for unsupervised medical image model discovery.

Anatomy Image Reconstruction +1

Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data

no code implementations7 Jul 2022 Yousef Yeganeh, Azade Farshad, Johann Boschmann, Richard Gaus, Maximilian Frantzen, Nassir Navab

Although most medical centers conduct similar medical imaging tasks, their differences, such as specializations, number of patients, and devices, lead to distinctive data distributions.

Clustering Federated Learning +3

Unsupervised Cross-Domain Feature Extraction for Single Blood Cell Image Classification

1 code implementation1 Jul 2022 Raheleh Salehi, Ario Sadafi, Armin Gruber, Peter Lienemann, Nassir Navab, Shadi Albarqouni, Carsten Marr

Here, we propose a cross-domain adapted autoencoder to extract features in an unsupervised manner on three different datasets of single white blood cells scanned from peripheral blood smears.

Image Classification

DeStripe: A Self2Self Spatio-Spectral Graph Neural Network with Unfolded Hessian for Stripe Artifact Removal in Light-sheet Microscopy

no code implementations27 Jun 2022 Yu Liu, Kurt Weiss, Nassir Navab, Carsten Marr, Jan Huisken, Tingying Peng

Light-sheet fluorescence microscopy (LSFM) is a cutting-edge volumetric imaging technique that allows for three-dimensional imaging of mesoscopic samples with decoupled illumination and detection paths.


U-PET: MRI-based Dementia Detection with Joint Generation of Synthetic FDG-PET Images

no code implementations16 Jun 2022 Marcel Kollovieh, Matthias Keicher, Stephan Wunderlich, Hendrik Burwinkel, Thomas Wendler, Nassir Navab

To this end, we propose a multi-task method based on U-Net that takes T1-weighted MR images as an input to generate synthetic FDG-PET images and classifies the dementia progression of the patient into cognitive normal (CN), cognitive impairment (MCI), and AD.

Virtual embeddings and self-consistency for self-supervised learning

no code implementations13 Jun 2022 Tariq Bdair, Hossam Abdelhamid, Nassir Navab, Shadi Albarqouni

We validate TriMix on eight benchmark datasets consisting of natural and medical images with an improvement of 2. 71% and 0. 41% better than the second-best models for both data types.

Data Augmentation Representation Learning +1

BFS-Net: Weakly Supervised Cell Instance Segmentation from Bright-Field Microscopy Z-Stacks

no code implementations9 Jun 2022 Shervin Dehghani, Benjamin Busam, Nassir Navab, Ali Nasseri

Despite its broad availability, volumetric information acquisition from Bright-Field Microscopy (BFM) is inherently difficult due to the projective nature of the acquisition process.

Instance Segmentation Semantic Segmentation

VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation

1 code implementation10 May 2022 Yuan Bi, Zhongliang Jiang, Yuan Gao, Thomas Wendler, Angelos Karlas, Nassir Navab

The results demonstrate that proposed approach can effectively and accurately navigate the probe towards the longitudinal view of vessels.

Navigate reinforcement-learning +1

Affective Medical Estimation and Decision Making via Visualized Learning and Deep Learning

1 code implementation9 May 2022 Mohammad Eslami, Solale Tabarestani, Ehsan Adeli, Glyn Elwyn, Tobias Elze, Mengyu Wang, Nazlee Zebardast, Nassir Navab, Malek Adjouadi

With the advent of sophisticated machine learning (ML) techniques and the promising results they yield, especially in medical applications, where they have been investigated for different tasks to enhance the decision-making process.

Decision Making Memorization

Y-Net: A Spatiospectral Dual-Encoder Networkfor Medical Image Segmentation

1 code implementation15 Apr 2022 Azade Farshad, Yousef Yeganeh, Peter Gehlbach, Nassir Navab

Automated segmentation of retinal optical coherence tomography (OCT) images has become an important recent direction in machine learning for medical applications.

 Ranked #1 on Retinal OCT Layer Segmentation on Duke SD-OCT (using extra training data)

Image Segmentation Medical Image Segmentation +3

Analyzing the Effects of Handling Data Imbalance on Learned Features from Medical Images by Looking Into the Models

no code implementations4 Apr 2022 Ashkan Khakzar, Yawei Li, Yang Zhang, Mirac Sanisoglu, Seong Tae Kim, Mina Rezaei, Bernd Bischl, Nassir Navab

One challenging property lurking in medical datasets is the imbalanced data distribution, where the frequency of the samples between the different classes is not balanced.

Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications

no code implementations1 Apr 2022 Kamilia Mullakaeva, Luca Cosmo, Anees Kazi, Seyed-Ahmad Ahmadi, Nassir Navab, Michael M. Bronstein

In this work, we propose Graph-in-Graph (GiG), a neural network architecture for protein classification and brain imaging applications that exploits the graph representation of the input data samples and their latent relation.

Property Prediction

FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays

no code implementations29 Mar 2022 Matthias Keicher, Kamilia Zaripova, Tobias Czempiel, Kristina Mach, Ashkan Khakzar, Nassir Navab

The automation of chest X-ray reporting has garnered significant interest due to the time-consuming nature of the task.

Intelligent Masking: Deep Q-Learning for Context Encoding in Medical Image Analysis

1 code implementation25 Mar 2022 Mojtaba Bahrami, Mahsa Ghorbani, Nassir Navab

We show that training the agent against the prediction model can significantly improve the semantic features extracted for downstream classification tasks.

Q-Learning Self-Supervised Learning

Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions

2 code implementations23 Mar 2022 Chantal Pellegrini, Anees Kazi, Nassir Navab

We test our method on two medical datasets of patient records, TADPOLE and MIMIC-III, including imaging and non-imaging features and different prediction tasks.

Disease Prediction Imputation +2

Conditional Generative Data Augmentation for Clinical Audio Datasets

no code implementations22 Mar 2022 Matthias Seibold, Armando Hoch, Mazda Farshad, Nassir Navab, Philipp Fürnstahl

In this work, we propose a novel data augmentation method for clinical audio datasets based on a conditional Wasserstein Generative Adversarial Network with Gradient Penalty (cWGAN-GP), operating on log-mel spectrograms.

Data Augmentation

4D-OR: Semantic Scene Graphs for OR Domain Modeling

1 code implementation22 Mar 2022 Ege Özsoy, Evin Pınar Örnek, Ulrich Eck, Tobias Czempiel, Federico Tombari, Nassir Navab

Towards this goal, for the first time, we propose using semantic scene graphs (SSG) to describe and summarize the surgical scene.

Scene Graph Generation

Surgical Workflow Recognition: from Analysis of Challenges to Architectural Study

no code implementations17 Mar 2022 Tobias Czempiel, Aidean Sharghi, Magdalini Paschali, Nassir Navab, Omid Mohareri

Algorithmic surgical workflow recognition is an ongoing research field and can be divided into laparoscopic (Internal) and operating room (External) analysis.

Know your sensORs -- A Modality Study For Surgical Action Classification

no code implementations16 Mar 2022 Lennart Bastian, Tobias Czempiel, Christian Heiliger, Konrad Karcz, Ulrich Eck, Benjamin Busam, Nassir Navab

Existing datasets from OR room cameras are thus far limited in size or modalities acquired, leaving it unclear which sensor modalities are best suited for tasks such as recognizing surgical action from videos.

Action Classification Action Recognition +1

GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting

2 code implementations CVPR 2022 Yan Di, Ruida Zhang, Zhiqiang Lou, Fabian Manhardt, Xiangyang Ji, Nassir Navab, Federico Tombari

While 6D object pose estimation has recently made a huge leap forward, most methods can still only handle a single or a handful of different objects, which limits their applications.

 Ranked #1 on 6D Pose Estimation on LineMOD (Mean ADD-S metric)

6D Pose Estimation 6D Pose Estimation using RGB +2

From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Prediction

no code implementations15 Mar 2022 Evin Pınar Örnek, Shristi Mudgal, Johanna Wald, Yida Wang, Nassir Navab, Federico Tombari

There have been numerous recently proposed methods for monocular depth prediction (MDP) coupled with the equally rapid evolution of benchmarking tools.

Benchmarking Depth Estimation +1

Transformers in Action: Weakly Supervised Action Segmentation

no code implementations14 Jan 2022 John Ridley, Huseyin Coskun, David Joseph Tan, Nassir Navab, Federico Tombari

The video action segmentation task is regularly explored under weaker forms of supervision, such as transcript supervision, where a list of actions is easier to obtain than dense frame-wise labels.

Action Segmentation

A Variational Bayesian Method for Similarity Learning in Non-Rigid Image Registration

1 code implementation CVPR 2022 Daniel Grzech, Mohammad Farid Azampour, Ben Glocker, Julia Schnabel, Nassir Navab, Bernhard Kainz, Loïc le Folgoc

We propose a novel variational Bayesian formulation for diffeomorphic non-rigid registration of medical images, which learns in an unsupervised way a data-specific similarity metric.

Image Registration

Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments

no code implementations7 Dec 2021 HyunJun Jung, Nikolas Brasch, Ales Leonardis, Nassir Navab, Benjamin Busam

Indirect Time-of-Flight (I-ToF) imaging is a widespread way of depth estimation for mobile devices due to its small size and affordable price.

Depth Estimation Depth Prediction

Object-aware Monocular Depth Prediction with Instance Convolutions

1 code implementation2 Dec 2021 Enis Simsar, Evin Pınar Örnek, Fabian Manhardt, Helisa Dhamo, Nassir Navab, Federico Tombari

With the advent of deep learning, estimating depth from a single RGB image has recently received a lot of attention, being capable of empowering many different applications ranging from path planning for robotics to computational cinematography.

Depth Estimation Depth Prediction +1

MIGS: Meta Image Generation from Scene Graphs

1 code implementation22 Oct 2021 Azade Farshad, Sabrina Musatian, Helisa Dhamo, Nassir Navab

We propose MIGS (Meta Image Generation from Scene Graphs), a meta-learning based approach for few-shot image generation from graphs that enables adapting the model to different scenes and increases the image quality by training on diverse sets of tasks.

Image Generation from Scene Graphs Meta-Learning +1

Semantic Image Alignment for Vehicle Localization

no code implementations8 Oct 2021 Markus Herb, Matthias Lemberger, Marcel M. Schmitt, Alexander Kurz, Tobias Weiherer, Nassir Navab, Federico Tombari

Accurate and reliable localization is a fundamental requirement for autonomous vehicles to use map information in higher-level tasks such as navigation or planning.

Autonomous Vehicles Semantic Segmentation +1

Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation

no code implementations24 Sep 2021 Mert Asim Karaoglu, Nikolas Brasch, Marijn Stollenga, Wolfgang Wein, Nassir Navab, Federico Tombari, Alexander Ladikos

The results of our experiments show that the proposed method improves the network's performance on real images by a considerable margin and can be employed in 3D reconstruction pipelines.

3D Reconstruction Depth Estimation

MetaMedSeg: Volumetric Meta-learning for Few-Shot Organ Segmentation

no code implementations18 Sep 2021 Anastasia Makarevich, Azade Farshad, Vasileios Belagiannis, Nassir Navab

In this work, we present MetaMedSeg, a gradient-based meta-learning algorithm that redefines the meta-learning task for the volumetric medical data with the goal to capture the variety between the slices.

Image Segmentation Medical Image Segmentation +3

Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs

1 code implementation ICCV 2021 Helisa Dhamo, Fabian Manhardt, Nassir Navab, Federico Tombari

Scene graphs are representations of a scene, composed of objects (nodes) and inter-object relationships (edges), proven to be particularly suited for this task, as they allow for semantic control on the generated content.

SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

2 code implementations ICCV 2021 Yan Di, Fabian Manhardt, Gu Wang, Xiangyang Ji, Nassir Navab, Federico Tombari

Directly regressing all 6 degrees-of-freedom (6DoF) for the object pose (e. g. the 3D rotation and translation) in a cluttered environment from a single RGB image is a challenging problem.

6D Pose Estimation 6D Pose Estimation using RGB +1

Unconditional Scene Graph Generation

no code implementations ICCV 2021 Sarthak Garg, Helisa Dhamo, Azade Farshad, Sabrina Musatian, Nassir Navab, Federico Tombari

Scene graphs, composed of nodes as objects and directed-edges as relationships among objects, offer an alternative representation of a scene that is more semantically grounded than images.

Anomaly Detection Graph Generation +2

Tracked 3D Ultrasound and Deep Neural Network-based Thyroid Segmentation reduce Interobserver Variability in Thyroid Volumetry

no code implementations10 Aug 2021 Markus Krönke, Christine Eilers, Desislava Dimova, Melanie Köhler, Gabriel Buschner, Lilit Mirzojan, Lemonia Konstantinidou, Marcus R. Makowski, James Nagarajah, Nassir Navab, Wolfgang Weber, Thomas Wendler

Conclusion: Tracked 3D ultrasound combined with a CNN segmentation significantly reduces interobserver variability in thyroid volumetry and increases the accuracy of the measurements with shorter acquisition times.

R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes

no code implementations10 Aug 2021 Stefano Gasperini, Patrick Koch, Vinzenz Dallabetta, Nassir Navab, Benjamin Busam, Federico Tombari

While self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches, violations of the static world assumption can still lead to erroneous depth predictions of traffic participants, posing a potential safety issue.

Autonomous Vehicles Monocular Depth Estimation

U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction

no code implementations29 Jul 2021 Matthias Keicher, Hendrik Burwinkel, David Bani-Harouni, Magdalini Paschali, Tobias Czempiel, Egon Burian, Marcus R. Makowski, Rickmer Braren, Nassir Navab, Thomas Wendler

Specifically, we introduce a multimodal similarity metric to build a population graph for clustering patients and an image-based end-to-end Graph Attention Network to process this graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation and mortality.

Clustering Decision Making +1

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

1 code implementation26 Jul 2021 Daniil Pakhomov, Sanchit Hira, Narayani Wagle, Kemar E. Green, Nassir Navab

Derived regions are consistent across different images and coincide with human-defined semantic classes on some datasets.

Image Segmentation Segmentation +1

Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images

no code implementations9 Jun 2021 Jakob Weiss, Nassir Navab

In this work, we introduce Deep Direct Volume Rendering (DeepDVR), a generalization of DVR that allows for the integration of deep neural networks into the DVR algorithm.

Colorization Inverse Rendering +1

Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedures

no code implementations9 Jun 2021 Ege Özsoy, Evin Pınar Örnek, Ulrich Eck, Federico Tombari, Nassir Navab

We then use MSSG to introduce a dynamically generated graphical user interface tool for surgical procedure analysis which could be used for many applications including process optimization, OR design and automatic report generation.

GKD: Semi-supervised Graph Knowledge Distillation for Graph-Independent Inference

1 code implementation8 Apr 2021 Mahsa Ghorbani, Mojtaba Bahrami, Anees Kazi, Mahdieh SoleymaniBaghshah, Hamid R. Rabiee, Nassir Navab

The soft pseudo-labels are then used to train a deep student network for disease prediction of unseen test data for which the graph modality is unavailable.

Disease Prediction graph construction +1

Towards Semantic Interpretation of Thoracic Disease and COVID-19 Diagnosis Models

1 code implementation4 Apr 2021 Ashkan Khakzar, Sabrina Musatian, Jonas Buchberger, Icxel Valeriano Quiroz, Nikolaus Pinger, Soroosh Baselizadeh, Seong Tae Kim, Nassir Navab

We present our findings using publicly available chest pathologies (CheXpert, NIH ChestX-ray8) and COVID-19 datasets (BrixIA, and COVID-19 chest X-ray segmentation dataset).

COVID-19 Diagnosis

IA-GCN: Interpretable Attention based Graph Convolutional Network for Disease prediction

no code implementations29 Mar 2021 Anees Kazi, Soroush Farghadani, Nassir Navab

The main novelty lies in the interpretable attention module (IAM), which directly operates on multi-modal features.

Decision Making Disease Prediction +2

GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images

no code implementations19 Mar 2021 Aadhithya Sankar, Matthias Keicher, Rami Eisawy, Abhijeet Parida, Franz Pfister, Seong Tae Kim, Nassir Navab

Disentangled representations can be useful in many downstream tasks, help to make deep learning models more interpretable, and allow for control over features of synthetically generated images that can be useful in training other models that require a large number of labelled or unlabelled data.


Longitudinal Quantitative Assessment of COVID-19 Infection Progression from Chest CTs

1 code implementation12 Mar 2021 Seong Tae Kim, Leili Goli, Magdalini Paschali, Ashkan Khakzar, Matthias Keicher, Tobias Czempiel, Egon Burian, Rickmer Braren, Nassir Navab, Thomas Wendler

Chest computed tomography (CT) has played an essential diagnostic role in assessing patients with COVID-19 by showing disease-specific image features such as ground-glass opacity and consolidation.

Computed Tomography (CT) COVID-19 Image Segmentation +2

OperA: Attention-Regularized Transformers for Surgical Phase Recognition

no code implementations5 Mar 2021 Tobias Czempiel, Magdalini Paschali, Daniel Ostler, Seong Tae Kim, Benjamin Busam, Nassir Navab

In this paper we introduce OperA, a transformer-based model that accurately predicts surgical phases from long video sequences.

Surgical phase recognition

Semi-Supervised Federated Peer Learning for Skin Lesion Classification

1 code implementation5 Mar 2021 Tariq Bdair, Nassir Navab, Shadi Albarqouni

With few annotated data, FedPerl is on par with a state-of-the-art method in skin lesion classification in the standard setup while outperforming SSFLs and the baselines by 1. 8% and 15. 8%, respectively.

Classification Federated Learning +4

RA-GCN: Graph Convolutional Network for Disease Prediction Problems with Imbalanced Data

1 code implementation27 Feb 2021 Mahsa Ghorbani, Anees Kazi, Mahdieh Soleymani Baghshah, Hamid R. Rabiee, Nassir Navab

This is accomplished by associating a graph-based neural network to each class, which is responsible for weighting the class samples and changing the importance of each sample for the classifier.

Disease Prediction Node Classification

Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation

1 code implementation20 Dec 2020 Haowen Deng, Mai Bui, Nassir Navab, Leonidas Guibas, Slobodan Ilic, Tolga Birdal

For the former we contributed our own dataset composed of five indoor scenes where it is unavoidable to capture images corresponding to views that are hard to uniquely identify.

Camera Relocalization Pose Estimation

An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation

1 code implementation6 Dec 2020 Roger D. Soberanis-Mukul, Nassir Navab, Shadi Albarqouni

In this context, we proposed a segmentation refinement method based on uncertainty analysis and graph convolutional networks.

Graph Learning Organ Segmentation +1

Rethinking Positive Aggregation and Propagation of Gradients in Gradient-based Saliency Methods

no code implementations1 Dec 2020 Ashkan Khakzar, Soroosh Baselizadeh, Nassir Navab

In this work, we empirically show that two approaches for handling the gradient information, namely positive aggregation, and positive propagation, break these methods.

Self-Supervised Out-of-Distribution Detection in Brain CT Scans

no code implementations10 Nov 2020 Abinav Ravi Venkatakrishnan, Seong Tae Kim, Rami Eisawy, Franz Pfister, Nassir Navab

To address these issues, recently, unsupervised deep anomaly detection methods that train the model on large-sized normal scans and detect abnormal scans by calculating reconstruction error have been reported.

Anomaly Detection Out-of-Distribution Detection +1

Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds

no code implementations28 Oct 2020 Stefano Gasperini, Mohammad-Ali Nikouei Mahani, Alvaro Marcos-Ramiro, Nassir Navab, Federico Tombari

Panoptic segmentation has recently unified semantic and instance segmentation, previously addressed separately, thus taking a step further towards creating more comprehensive and efficient perception systems.

Clustering Instance Segmentation +2

SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion

2 code implementations26 Oct 2020 Shun-Cheng Wu, Keisuke Tateno, Nassir Navab, Federico Tombari

We propose a framework that ameliorates this issue by performing scene reconstruction and semantic scene completion jointly in an incremental and real-time manner, based on an input sequence of depth maps.

3D Semantic Scene Completion

RGB-D SLAM with Structural Regularities

1 code implementation15 Oct 2020 Yanyan Li, Raza Yunus, Nikolas Brasch, Nassir Navab, Federico Tombari

This work proposes a RGB-D SLAM system specifically designed for structured environments and aimed at improved tracking and mapping accuracy by relying on geometric features that are extracted from the surrounding.