Search Results for author: Mathias Unberath

Found 67 papers, 25 papers with code

FastSAM3D: An Efficient Segment Anything Model for 3D Volumetric Medical Images

1 code implementation14 Mar 2024 Yiqing Shen, Jingxing Li, Xinyuan Shao, Blanca Inigo Romillo, Ankush Jindal, David Dreizin, Mathias Unberath

Segment anything models (SAMs) are gaining attention for their zero-shot generalization capability in segmenting objects of unseen classes and in unseen domains when properly prompted.

3D Medical Imaging Segmentation Transfer Learning +1

FluoroSAM: A Language-aligned Foundation Model for X-ray Image Segmentation

1 code implementation12 Mar 2024 Benjamin D. Killeen, Liam J. Wang, Han Zhang, Mehran Armand, Russell H. Taylor, Dave Dreizin, Greg Osgood, Mathias Unberath

Recently, foundation models (FMs) -- machine learning models trained on large amounts of highly variable data thus enabling broad applicability -- have emerged as promising tools for automated image analysis.

Image Segmentation Semantic Segmentation +1

From Generalization to Precision: Exploring SAM for Tool Segmentation in Surgical Environments

no code implementations28 Feb 2024 Kanyifeechukwu J. Oguine, Roger D. Soberanis-Mukul, Nathan Drenkow, Mathias Unberath

We argue that SAM drastically over-segment images with high corruption levels, resulting in degraded performance when only a single segmentation mask is considered, while the combination of the masks overlapping the object of interest generates an accurate prediction.

Segmentation Zero Shot Segmentation

An Endoscopic Chisel: Intraoperative Imaging Carves 3D Anatomical Models

no code implementations19 Feb 2024 Jan Emily Mangulabnan, Roger D. Soberanis-Mukul, Timo Teufel, Manish Sahu, Jose L. Porras, S. Swaroop Vedula, Masaru Ishii, Gregory Hager, Russell H. Taylor, Mathias Unberath

Purpose: Preoperative imaging plays a pivotal role in sinus surgery where CTs offer patient-specific insights of complex anatomy, enabling real-time intraoperative navigation to complement endoscopy imaging.

Anatomy Monocular Depth Estimation

Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review

no code implementations30 Oct 2023 Catalina Gomez, Sue Min Cho, Shichang Ke, Chien-Ming Huang, Mathias Unberath

However, how the information is presented, e. g., the sequence of recommendations and solicitation of interpretations, is equally crucial as complex interactions may emerge between humans and AI.

Decision Making

A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video

no code implementations22 Oct 2023 Jan Emily Mangulabnan, Roger D. Soberanis-Mukul, Timo Teufel, Isabela Hernández, Jonas Winter, Manish Sahu, Jose L. Porras, S. Swaroop Vedula, Masaru Ishii, Gregory Hager, Russell H. Taylor, Mathias Unberath

In this work, we perform a quantitative analysis of a self-supervised approach for sinus reconstruction using endoscopic sequences paired with optical tracking and high-resolution computed tomography acquired from nine ex-vivo specimens.

3D Reconstruction Anatomy +3

RobustCLEVR: A Benchmark and Framework for Evaluating Robustness in Object-centric Learning

no code implementations28 Aug 2023 Nathan Drenkow, Mathias Unberath

Lastly, while conventional robustness evaluations view corruptions as out-of-distribution, we use our causal framework to show that even training on in-distribution image corruptions does not guarantee increased model robustness.

Image Generation Object +1

TransNuSeg: A Lightweight Multi-Task Transformer for Nuclei Segmentation

1 code implementation16 Jul 2023 Zhenqi He, Mathias Unberath, Jing Ke, Yiqing Shen

In conclusion, TransNuSeg confirms the strength of Transformer in the context of nuclei segmentation, which thus can serve as an efficient solution for real clinical practice.

Multi-Task Learning Segmentation

Automated Artifact Detection in Ultra-widefield Fundus Photography of Patients with Sickle Cell Disease

no code implementations11 Jul 2023 Anqi Feng, Dimitri Johnson, Grace R. Reilly, Loka Thangamathesvaran, Ann Nampomba, Mathias Unberath, Adrienne W. Scott, Craig Jones

A pre-trained ResNet-50 neural network was trained on a subset of the images and the classification accuracy, sensitivity, and specificity were quantified on the hold out test set.

Artifact Detection Specificity

Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models

no code implementations6 Apr 2023 Mitchell Pavlak, Nathan Drenkow, Nicholas Petrick, Mohammad Mehdi Farhangi, Mathias Unberath

To safely deploy deep learning-based computer vision models for computer-aided detection and diagnosis, we must ensure that they are robust and reliable.

Attribute Causal Inference +1

MoViT: Memorizing Vision Transformers for Medical Image Analysis

no code implementations27 Mar 2023 Yiqing Shen, Pengfei Guo, Jingpu Wu, Qianqi Huang, Nhat Le, Jinyuan Zhou, Shanshan Jiang, Mathias Unberath

We evaluate our method on a public histology image dataset and an in-house MRI dataset, demonstrating that MoViT applied to varied medical image analysis tasks, can outperform vanilla transformer models across varied data regimes, especially in cases where only a small amount of annotated data is available.

Decision Making Inductive Bias

Rethinking Causality-driven Robot Tool Segmentation with Temporal Constraints

1 code implementation30 Nov 2022 Hao Ding, Jie Ying Wu, Zhaoshuo Li, Mathias Unberath

Method: To address the above limitations, we take temporal relation into consideration and propose a temporal causal model for robot tool segmentation on video sequences.

counterfactual Segmentation

Context-Enhanced Stereo Transformer

1 code implementation21 Oct 2022 Weiyu Guo, Zhaoshuo Li, Yongkui Yang, Zheng Wang, Russell H. Taylor, Mathias Unberath, Alan Yuille, Yingwei Li

We construct our stereo depth estimation model, Context Enhanced Stereo Transformer (CSTR), by plugging CEP into the state-of-the-art stereo depth estimation method Stereo Transformer.

Stereo Depth Estimation Stereo Matching

SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In Silico Experiments

1 code implementation13 Jun 2022 Cong Gao, Benjamin D. Killeen, Yicheng Hu, Robert B. Grupp, Russell H. Taylor, Mehran Armand, Mathias Unberath

Here, we demonstrate that creating realistic simulated images from human models is a viable alternative and complement to large-scale in situ data collection.

Domain Generalization Lesion Segmentation

CaRTS: Causality-driven Robot Tool Segmentation from Vision and Kinematics Data

1 code implementation15 Mar 2022 Hao Ding, Jintan Zhang, Peter Kazanzides, Jie Ying Wu, Mathias Unberath

Vision-based segmentation of the robotic tool during robot-assisted surgery enables downstream applications, such as augmented reality feedback, while allowing for inaccuracies in robot kinematics.

counterfactual Segmentation

SAGE: SLAM with Appearance and Geometry Prior for Endoscopy

1 code implementation19 Feb 2022 Xingtong Liu, Zhaoshuo Li, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

In endoscopy, many applications (e. g., surgical navigation) would benefit from a real-time method that can simultaneously track the endoscope and reconstruct the dense 3D geometry of the observed anatomy from a monocular endoscopic video.

Anatomy Simultaneous Localization and Mapping

Mapping DNN Embedding Manifolds for Network Generalization Prediction

no code implementations3 Feb 2022 Molly O'Brien, Julia Bukowski, Mathias Unberath, Aria Pezeshk, Greg Hager

Understanding Deep Neural Network (DNN) performance in changing conditions is essential for deploying DNNs in safety critical applications with unconstrained environments, e. g., perception for self-driving vehicles or medical image analysis.

Explainable Medical Imaging AI Needs Human-Centered Design: Guidelines and Evidence from a Systematic Review

no code implementations21 Dec 2021 Haomin Chen, Catalina Gomez, Chien-Ming Huang, Mathias Unberath

To alleviate these shortcomings in forthcoming research while acknowledging the challenges of human-centered design in healthcare, we introduce the INTRPRT guideline, a systematic design directive for transparent ML systems in medical image analysis.

A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?

no code implementations1 Dec 2021 Nathan Drenkow, Numair Sani, Ilya Shpitser, Mathias Unberath

We find this area of research has received disproportionately less attention relative to adversarial machine learning, yet a significant robustness gap exists that manifests in performance degradation similar in magnitude to adversarial conditions.

Adversarial Robustness Data Augmentation +1

Temporally Consistent Online Depth Estimation in Dynamic Scenes

no code implementations17 Nov 2021 Zhaoshuo Li, Wei Ye, Dilin Wang, Francis X. Creighton, Russell H. Taylor, Ganesh Venkatesh, Mathias Unberath

We present a framework named Consistent Online Dynamic Depth (CODD) to produce temporally consistent depth estimates in dynamic scenes in an online setting.

Stereo Depth Estimation

Pediatric Otoscopy Video Screening with Shift Contrastive Anomaly Detection

no code implementations25 Oct 2021 Weiyao Wang, Aniruddha Tamhane, Christine Santos, John R. Rzasa, James H. Clark, Therese L. Canares, Mathias Unberath

Our method achieves an AUROC of 88. 0% on the patient-level and also outperforms the average of a group of 25 clinicians in a comparative study, which is the largest of such published to date.

Anomaly Detection

On the Sins of Image Synthesis Loss for Self-supervised Depth Estimation

no code implementations13 Sep 2021 Zhaoshuo Li, Nathan Drenkow, Hao Ding, Andy S. Ding, Alexander Lu, Francis X. Creighton, Russell H. Taylor, Mathias Unberath

It is based on the idea that observed frames can be synthesized from neighboring frames if accurate depth of the scene is known - or in this case, estimated.

Attribute Depth Estimation +3

An Interpretable Algorithm for Uveal Melanoma Subtyping from Whole Slide Cytology Images

no code implementations13 Aug 2021 Haomin Chen, T. Y. Alvin Liu, Catalina Gomez, Zelia Correa, Mathias Unberath

Algorithmic decision support is rapidly becoming a staple of personalized medicine, especially for high-stakes recommendations in which access to certain information can drastically alter the course of treatment, and thus, patient outcome; a prominent example is radiomics for cancer subtyping.

E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception

2 code implementations1 Jul 2021 Yonghao Long, Zhaoshuo Li, Chi Hang Yee, Chi Fai Ng, Russell H. Taylor, Mathias Unberath, Qi Dou

After that, a dynamic reconstruction algorithm which can estimate the tissue deformation and camera movement, and aggregate the information over time is proposed for surgical scene reconstruction.

Anatomy Depth Estimation +1

Neighborhood Normalization for Robust Geometric Feature Learning

1 code implementation CVPR 2021 Xingtong Liu, Benjamin D. Killeen, Ayushi Sinha, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

Extracting geometric features from 3D models is a common first step in applications such as 3D registration, tracking, and scene flow estimation.

Scene Flow Estimation

An Interpretable Approach to Automated Severity Scoring in Pelvic Trauma

no code implementations21 May 2021 Anna Zapaishchykova, David Dreizin, Zhaoshuo Li, Jie Ying Wu, Shahrooz Faghih Roohi, Mathias Unberath

The method operates similarly to human interpretation of CT scans and first detects distinct pelvic fractures on CT with high specificity using a Faster-RCNN model that are then interpreted using a structural causal model based on clinical best practices to infer an initial Tile grade.

Specificity

Multimodal and self-supervised representation learning for automatic gesture recognition in surgical robotics

no code implementations31 Oct 2020 Aniruddha Tamhane, Jie Ying Wu, Mathias Unberath

We develop a self-supervised, multi-modal representation learning paradigm that learns representations for surgical gestures from video and kinematics.

Gesture Recognition Representation Learning +1

A Learning-based Method for Online Adjustment of C-arm Cone-Beam CT Source Trajectories for Artifact Avoidance

no code implementations14 Aug 2020 Mareike Thies, Jan-Nico Zäch, Cong Gao, Russell Taylor, Nassir Navab, Andreas Maier, Mathias Unberath

We propose to adjust the C-arm CBCT source trajectory during the scan to optimize reconstruction quality with respect to a certain task, i. e. verification of screw placement.

Anatomy Tomographic Reconstructions

Augment Yourself: Mixed Reality Self-Augmentation Using Optical See-through Head-mounted Displays and Physical Mirrors

no code implementations6 Jul 2020 Mathias Unberath, Kevin Yu, Roghayeh Barmaki, Alex Johnson, Nassir Navab

Consequently, most MR applications that are centered around the user, such as virtual dressing rooms or learning of body movements, cannot be realized with HMDs.

Anatomy Mixed Reality

Artificial Intelligence-based Clinical Decision Support for COVID-19 -- Where Art Thou?

no code implementations5 Jun 2020 Mathias Unberath, Kimia Ghobadi, Scott Levin, Jeremiah Hinson, Gregory D. Hager

The COVID-19 crisis has brought about new clinical questions, new workflows, and accelerated distributed healthcare needs.

A County-level Dataset for Informing the United States' Response to COVID-19

1 code implementation1 Apr 2020 Benjamin D. Killeen, Jie Ying Wu, Kinjal Shah, Anna Zapaishchykova, Philipp Nikutta, Aniruddha Tamhane, Shreya Chakraborty, Jinchi Wei, Tiger Gao, Mareike Thies, Mathias Unberath

As the coronavirus disease 2019 (COVID-19) becomes a global pandemic, policy makers must enact interventions to stop its spread.

Computers and Society Databases Physics and Society Populations and Evolution

Generalizing Spatial Transformers to Projective Geometry with Applications to 2D/3D Registration

1 code implementation24 Mar 2020 Cong Gao, Xingtong Liu, Wenhao Gu, Benjamin Killeen, Mehran Armand, Russell Taylor, Mathias Unberath

We propose a novel Projective Spatial Transformer module that generalizes spatial transformers to projective geometry, thus enabling differentiable volume rendering.

Anatomy

Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment

1 code implementation18 Mar 2020 Xingtong Liu, Maia Stiber, Jindan Huang, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

Reconstructing accurate 3D surface models of sinus anatomy directly from an endoscopic video is a promising avenue for cross-sectional and longitudinal analysis to better understand the relationship between sinus anatomy and surgical outcomes.

3D Reconstruction Anatomy

From Perspective X-ray Imaging to Parallax-Robust Orthographic Stitching

no code implementations5 Mar 2020 Javad Fotouhi, Xingtong Liu, Mehran Armand, Nassir Navab, Mathias Unberath

Stitching images acquired under perspective projective geometry is a relevant topic in computer vision with multiple applications ranging from smartphone panoramas to the construction of digital maps.

Anatomy Image Stitching

Spatiotemporal-Aware Augmented Reality: Redefining HCI in Image-Guided Therapy

no code implementations4 Mar 2020 Javad Fotouhi, Arian Mehrfard, Tianyu Song, Alex Johnson, Greg Osgood, Mathias Unberath, Mehran Armand, Nassir Navab

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies.

Anatomy Management

Extremely Dense Point Correspondences using a Learned Feature Descriptor

1 code implementation CVPR 2020 Xingtong Liu, Yiping Zheng, Benjamin Killeen, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

In direct comparison to recent local and dense descriptors on an in-house sinus endoscopy dataset, we demonstrate that our proposed dense descriptor can generalize to unseen patients and scopes, thereby largely improving the performance of Structure from Motion (SfM) in terms of model density and completeness.

3D Reconstruction Anatomy +2

Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration

1 code implementation16 Nov 2019 Robert Grupp, Mathias Unberath, Cong Gao, Rachel Hegeman, Ryan Murphy, Clayton Alexander, Yoshito Otake, Benjamin McArthur, Mehran Armand, Russell Taylor

By using these annotations as training data for neural networks, state of the art performance in fluoroscopic segmentation and landmark localization was achieved.

Anatomy

Fast and Automatic Periacetabular Osteotomy Fragment Pose Estimation Using Intraoperatively Implanted Fiducials and Single-View Fluoroscopy

no code implementations22 Oct 2019 Robert Grupp, Ryan Murphy, Rachel Hegeman, Clayton Alexander, Mathias Unberath, Yoshito Otake, Benjamin McArthur, Mehran Armand, Russell Taylor

The relative pose of the fragment is established by estimating the movement of the two BB constellations using a single fluoroscopic view taken after osteotomy and fragment relocation.

Pose Estimation

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

no code implementations20 Oct 2019 Tom Vercauteren, Mathias Unberath, Nicolas Padoy, Nassir Navab

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice.

Decision Making

Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories

no code implementations19 Sep 2019 Jan-Nico Zaech, Cong Gao, Bastian Bier, Russell Taylor, Andreas Maier, Nassir Navab, Mathias Unberath

Metal artifacts in computed tomography (CT) arise from a mismatch between physics of image formation and idealized assumptions during tomographic reconstruction.

Computed Tomography (CT)

Self-supervised Dense 3D Reconstruction from Monocular Endoscopic Video

no code implementations6 Sep 2019 Xingtong Liu, Ayushi Sinha, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

We present a self-supervised learning-based pipeline for dense 3D reconstruction from full-length monocular endoscopic videos without a priori modeling of anatomy or shading.

3D Reconstruction Anatomy +1

Reflective-AR Display: An Interaction Methodology for Virtual-Real Alignment in Medical Robotics

no code implementations23 Jul 2019 Javad Fotouhi, Tianyu Song, Arian Mehrfard, Giacomo Taylor, Qiaochu Wang, Fengfang Xian, Alejandro Martin-Gomez, Bernhard Fuerst, Mehran Armand, Mathias Unberath, Nassir Navab

To overcome this challenge, we introduce a novel registration concept for intuitive alignment of AR content to its physical counterpart by providing a multi-view AR experience via reflective-AR displays that simultaneously show the augmentations from multiple viewpoints.

LumiPath -- Towards Real-time Physically-based Rendering on Embedded Devices

1 code implementation9 Mar 2019 Laura Fink, Sing Chun Lee, Jie Ying Wu, Xingtong Liu, Tianyu Song, Yordanka Stoyanova, Marc Stamminger, Nassir Navab, Mathias Unberath

With the increasing computational power of today's workstations, real-time physically-based rendering is within reach, rapidly gaining attention across a variety of domains.

Data Visualization Image Generation +1

Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods

1 code implementation20 Feb 2019 Xingtong Liu, Ayushi Sinha, Masaru Ishii, Gregory D. Hager, Austin Reiter, Russell H. Taylor, Mathias Unberath

We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading.

Anatomy Computed Tomography (CT) +2

Localizing dexterous surgical tools in X-ray for image-based navigation

2 code implementations20 Jan 2019 Cong Gao, Mathias Unberath, Russell Taylor, Mehran Armand

This manuscript describes a first step towards leveraging semantic information of the imaged object to initialize 2D/3D registration within the capture range of image-based registration by performing concurrent segmentation and localization of dexterous surgical tools in X-ray images.

Retrieval

Learning to See Forces: Surgical Force Prediction with RGB-Point Cloud Temporal Convolutional Networks

no code implementations31 Jul 2018 Cong Gao, Xingtong Liu, Michael Peven, Mathias Unberath, Austin Reiter

Our method results in a mean absolute error of 0. 814 N in the ex vivo study, suggesting that it may be a promising alternative to hardware based surgical force feedback in endoscopic procedures.

Self-supervised Learning for Dense Depth Estimation in Monocular Endoscopy

no code implementations25 Jun 2018 Xingtong Liu, Ayushi Sinha, Mathias Unberath, Masaru Ishii, Gregory Hager, Russell H. Taylor, Austin Reiter

We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading.

Anatomy Depth Estimation +2

Augmented Reality-based Feedback for Technician-in-the-loop C-arm Repositioning

no code implementations22 Jun 2018 Mathias Unberath, Javad Fotouhi, Jonas Hajek, Andreas Maier, Greg Osgood, Russell Taylor, Mehran Armand, Nassir Navab

For C-arm repositioning to a particular target view, the recorded C-arm pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician.

Anatomy

Exploiting Partial Structural Symmetry For Patient-Specific Image Augmentation in Trauma Interventions

no code implementations9 Apr 2018 Javad Fotouhi, Mathias Unberath, Giacomo Taylor, Arash Ghaani Farashahi, Bastian Bier, Russell H. Taylor, Greg M. Osgood, M. D., Mehran Armand, Nassir Navab

The main challenge is to automatically estimate the desired plane of symmetry within the patient's pre-operative CT. We propose to estimate this plane using a non-linear optimization strategy, by minimizing Tukey's biweight robust estimator, relying on the partial symmetry of the anatomy.

Anatomy Image Augmentation

X-ray-transform Invariant Anatomical Landmark Detection for Pelvic Trauma Surgery

2 code implementations22 Mar 2018 Bastian Bier, Mathias Unberath, Jan-Nico Zaech, Javad Fotouhi, Mehran Armand, Greg Osgood, Nassir Navab, Andreas Maier

In this work, we present a method to automatically detect anatomical landmarks in X-ray images independent of the viewing direction.

Anatomy Decision Making +1

Cannot find the paper you are looking for? You can Submit a new open access paper.