Search Results for author: Mathias Unberath

Found 41 papers, 13 papers with code

On the Sins of Image Synthesis Loss for Self-supervised Depth Estimation

no code implementations13 Sep 2021 Zhaoshuo Li, Nathan Drenkow, Hao Ding, Andy S. Ding, Alexander Lu, Francis X. Creighton, Russell H. Taylor, Mathias Unberath

It is based on the idea that observed frames can be synthesized from neighboring frames if accurate depth of the scene is known - or in this case, estimated.

Depth Estimation Image Generation +2

An Interpretable Algorithm for Uveal Melanoma Subtyping from Whole Slide Cytology Images

no code implementations13 Aug 2021 Haomin Chen, T. Y. Alvin Liu, Catalina Gomez, Zelia Correa, Mathias Unberath

Algorithmic decision support is rapidly becoming a staple of personalized medicine, especially for high-stakes recommendations in which access to certain information can drastically alter the course of treatment, and thus, patient outcome; a prominent example is radiomics for cancer subtyping.

E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception

no code implementations1 Jul 2021 Yonghao Long, Zhaoshuo Li, Chi Hang Yee, Chi Fai Ng, Russell H. Taylor, Mathias Unberath, Qi Dou

After that, a dynamic reconstruction algorithm which can estimate the tissue deformation and camera movement, and aggregate the information over time is proposed for surgical scene reconstruction.

Depth Estimation

Neighborhood Normalization for Robust Geometric Feature Learning

1 code implementation CVPR 2021 Xingtong Liu, Benjamin D. Killeen, Ayushi Sinha, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

Extracting geometric features from 3D models is a common first step in applications such as 3D registration, tracking, and scene flow estimation.

Scene Flow Estimation

An Interpretable Approach to Automated Severity Scoring in Pelvic Trauma

no code implementations21 May 2021 Anna Zapaishchykova, David Dreizin, Zhaoshuo Li, Jie Ying Wu, Shahrooz Faghih Roohi, Mathias Unberath

The method operates similarly to human interpretation of CT scans and first detects distinct pelvic fractures on CT with high specificity using a Faster-RCNN model that are then interpreted using a structural causal model based on clinical best practices to infer an initial Tile grade.

Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration

no code implementations14 Nov 2020 Matthias Grimm, Javier Esteban, Mathias Unberath, Nassir Navab

First, a neural network is trained once to detect a set of anatomical landmarks on simulated X-rays.

Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers

1 code implementation5 Nov 2020 Zhaoshuo Li, Xingtong Liu, Nathan Drenkow, Andy Ding, Francis X. Creighton, Russell H. Taylor, Mathias Unberath

Stereo depth estimation relies on optimal correspondence matching between pixels on epipolar lines in the left and right images to infer depth.

Stereo Depth Estimation

Multimodal and self-supervised representation learning for automatic gesture recognition in surgical robotics

no code implementations31 Oct 2020 Aniruddha Tamhane, Jie Ying Wu, Mathias Unberath

We develop a self-supervised, multi-modal representation learning paradigm that learns representations for surgical gestures from video and kinematics.

Gesture Recognition Representation Learning +1

A Learning-based Method for Online Adjustment of C-arm Cone-Beam CT Source Trajectories for Artifact Avoidance

no code implementations14 Aug 2020 Mareike Thies, Jan-Nico Zäch, Cong Gao, Russell Taylor, Nassir Navab, Andreas Maier, Mathias Unberath

We propose to adjust the C-arm CBCT source trajectory during the scan to optimize reconstruction quality with respect to a certain task, i. e. verification of screw placement.

Tomographic Reconstructions

Augment Yourself: Mixed Reality Self-Augmentation Using Optical See-through Head-mounted Displays and Physical Mirrors

no code implementations6 Jul 2020 Mathias Unberath, Kevin Yu, Roghayeh Barmaki, Alex Johnson, Nassir Navab

Consequently, most MR applications that are centered around the user, such as virtual dressing rooms or learning of body movements, cannot be realized with HMDs.

Mixed Reality

Artificial Intelligence-based Clinical Decision Support for COVID-19 -- Where Art Thou?

no code implementations5 Jun 2020 Mathias Unberath, Kimia Ghobadi, Scott Levin, Jeremiah Hinson, Gregory D. Hager

The COVID-19 crisis has brought about new clinical questions, new workflows, and accelerated distributed healthcare needs.

A County-level Dataset for Informing the United States' Response to COVID-19

1 code implementation1 Apr 2020 Benjamin D. Killeen, Jie Ying Wu, Kinjal Shah, Anna Zapaishchykova, Philipp Nikutta, Aniruddha Tamhane, Shreya Chakraborty, Jinchi Wei, Tiger Gao, Mareike Thies, Mathias Unberath

As the coronavirus disease 2019 (COVID-19) becomes a global pandemic, policy makers must enact interventions to stop its spread.

Computers and Society Databases Physics and Society Populations and Evolution

Generalizing Spatial Transformers to Projective Geometry with Applications to 2D/3D Registration

1 code implementation24 Mar 2020 Cong Gao, Xingtong Liu, Wenhao Gu, Benjamin Killeen, Mehran Armand, Russell Taylor, Mathias Unberath

We propose a novel Projective Spatial Transformer module that generalizes spatial transformers to projective geometry, thus enabling differentiable volume rendering.

Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment

1 code implementation18 Mar 2020 Xingtong Liu, Maia Stiber, Jindan Huang, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

Reconstructing accurate 3D surface models of sinus anatomy directly from an endoscopic video is a promising avenue for cross-sectional and longitudinal analysis to better understand the relationship between sinus anatomy and surgical outcomes.

3D Reconstruction Structure from Motion

From Perspective X-ray Imaging to Parallax-Robust Orthographic Stitching

no code implementations5 Mar 2020 Javad Fotouhi, Xingtong Liu, Mehran Armand, Nassir Navab, Mathias Unberath

Stitching images acquired under perspective projective geometry is a relevant topic in computer vision with multiple applications ranging from smartphone panoramas to the construction of digital maps.

Image Stitching

Spatiotemporal-Aware Augmented Reality: Redefining HCI in Image-Guided Therapy

no code implementations4 Mar 2020 Javad Fotouhi, Arian Mehrfard, Tianyu Song, Alex Johnson, Greg Osgood, Mathias Unberath, Mehran Armand, Nassir Navab

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies.

Extremely Dense Point Correspondences using a Learned Feature Descriptor

1 code implementation CVPR 2020 Xingtong Liu, Yiping Zheng, Benjamin Killeen, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

In direct comparison to recent local and dense descriptors on an in-house sinus endoscopy dataset, we demonstrate that our proposed dense descriptor can generalize to unseen patients and scopes, thereby largely improving the performance of Structure from Motion (SfM) in terms of model density and completeness.

3D Reconstruction Optical Flow Estimation +1

Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration

1 code implementation16 Nov 2019 Robert Grupp, Mathias Unberath, Cong Gao, Rachel Hegeman, Ryan Murphy, Clayton Alexander, Yoshito Otake, Benjamin McArthur, Mehran Armand, Russell Taylor

By using these annotations as training data for neural networks, state of the art performance in fluoroscopic segmentation and landmark localization was achieved.

Fast and Automatic Periacetabular Osteotomy Fragment Pose Estimation Using Intraoperatively Implanted Fiducials and Single-View Fluoroscopy

no code implementations22 Oct 2019 Robert Grupp, Ryan Murphy, Rachel Hegeman, Clayton Alexander, Mathias Unberath, Yoshito Otake, Benjamin McArthur, Mehran Armand, Russell Taylor

The relative pose of the fragment is established by estimating the movement of the two BB constellations using a single fluoroscopic view taken after osteotomy and fragment relocation.

Pose Estimation

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

no code implementations20 Oct 2019 Tom Vercauteren, Mathias Unberath, Nicolas Padoy, Nassir Navab

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice.

Decision Making

Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories

no code implementations19 Sep 2019 Jan-Nico Zaech, Cong Gao, Bastian Bier, Russell Taylor, Andreas Maier, Nassir Navab, Mathias Unberath

Metal artifacts in computed tomography (CT) arise from a mismatch between physics of image formation and idealized assumptions during tomographic reconstruction.

Computed Tomography (CT)

Self-supervised Dense 3D Reconstruction from Monocular Endoscopic Video

no code implementations6 Sep 2019 Xingtong Liu, Ayushi Sinha, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

We present a self-supervised learning-based pipeline for dense 3D reconstruction from full-length monocular endoscopic videos without a priori modeling of anatomy or shading.

3D Reconstruction Self-Supervised Learning

Reflective-AR Display: An Interaction Methodology for Virtual-Real Alignment in Medical Robotics

no code implementations23 Jul 2019 Javad Fotouhi, Tianyu Song, Arian Mehrfard, Giacomo Taylor, Qiaochu Wang, Fengfang Xian, Alejandro Martin-Gomez, Bernhard Fuerst, Mehran Armand, Mathias Unberath, Nassir Navab

To overcome this challenge, we introduce a novel registration concept for intuitive alignment of AR content to its physical counterpart by providing a multi-view AR experience via reflective-AR displays that simultaneously show the augmentations from multiple viewpoints.

LumiPath -- Towards Real-time Physically-based Rendering on Embedded Devices

1 code implementation9 Mar 2019 Laura Fink, Sing Chun Lee, Jie Ying Wu, Xingtong Liu, Tianyu Song, Yordanka Stoyanova, Marc Stamminger, Nassir Navab, Mathias Unberath

With the increasing computational power of today's workstations, real-time physically-based rendering is within reach, rapidly gaining attention across a variety of domains.

Data Visualization Image Generation

Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods

1 code implementation20 Feb 2019 Xingtong Liu, Ayushi Sinha, Masaru Ishii, Gregory D. Hager, Austin Reiter, Russell H. Taylor, Mathias Unberath

We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading.

Computed Tomography (CT) Depth Estimation +2

Localizing dexterous surgical tools in X-ray for image-based navigation

2 code implementations20 Jan 2019 Cong Gao, Mathias Unberath, Russell Taylor, Mehran Armand

This manuscript describes a first step towards leveraging semantic information of the imaged object to initialize 2D/3D registration within the capture range of image-based registration by performing concurrent segmentation and localization of dexterous surgical tools in X-ray images.

Learning to See Forces: Surgical Force Prediction with RGB-Point Cloud Temporal Convolutional Networks

no code implementations31 Jul 2018 Cong Gao, Xingtong Liu, Michael Peven, Mathias Unberath, Austin Reiter

Our method results in a mean absolute error of 0. 814 N in the ex vivo study, suggesting that it may be a promising alternative to hardware based surgical force feedback in endoscopic procedures.

Self-supervised Learning for Dense Depth Estimation in Monocular Endoscopy

no code implementations25 Jun 2018 Xingtong Liu, Ayushi Sinha, Mathias Unberath, Masaru Ishii, Gregory Hager, Russell H. Taylor, Austin Reiter

We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading.

Depth Estimation Self-Supervised Learning +1

Augmented Reality-based Feedback for Technician-in-the-loop C-arm Repositioning

no code implementations22 Jun 2018 Mathias Unberath, Javad Fotouhi, Jonas Hajek, Andreas Maier, Greg Osgood, Russell Taylor, Mehran Armand, Nassir Navab

For C-arm repositioning to a particular target view, the recorded C-arm pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician.

Exploiting Partial Structural Symmetry For Patient-Specific Image Augmentation in Trauma Interventions

no code implementations9 Apr 2018 Javad Fotouhi, Mathias Unberath, Giacomo Taylor, Arash Ghaani Farashahi, Bastian Bier, Russell H. Taylor, Greg M. Osgood, M. D., Mehran Armand, Nassir Navab

The main challenge is to automatically estimate the desired plane of symmetry within the patient's pre-operative CT. We propose to estimate this plane using a non-linear optimization strategy, by minimizing Tukey's biweight robust estimator, relying on the partial symmetry of the anatomy.

Image Augmentation

DeepDRR -- A Catalyst for Machine Learning in Fluoroscopy-guided Procedures

2 code implementations22 Mar 2018 Mathias Unberath, Jan-Nico Zaech, Sing Chun Lee, Bastian Bier, Javad Fotouhi, Mehran Armand, Nassir Navab

Machine learning-based approaches outperform competing methods in most disciplines relevant to diagnostic radiology.

Domain Adaptation

Plan in 2D, execute in 3D: An augmented reality solution for cup placement in total hip arthroplasty

no code implementations4 Jan 2018 Javad Fotouhi, Clayton P. Alexander, Mathias Unberath, Giacomo Taylor, Sing Chun Lee, Bernhard Fuerst, Alex Johnson, Greg Osgood, Russell H. Taylor, Harpal Khanuja, Mehran Armand, Nassir Navab

Reproducibly achieving proper implant alignment is a critical step in total hip arthroplasty (THA) procedures that has been shown to substantially affect patient outcome.

Cannot find the paper you are looking for? You can Submit a new open access paper.