Search Results for author:
Found 412 papers, 143 papers with code
Adversarial Wear and Tear: Exploiting Natural Damage for Generating Physical-World Adversarial Examples
The translation network encodes the characteristics of damaged signs into a latent `damage style code'.
Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior
Recent advancements in large language models (LLMs) have demonstrated that fine-tuning and human alignment can render LLMs harmless.
Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Cameras
Additionally, we propose a 3D scene reconstruction pipeline that optimizes depth maps' scales, shifts, and a few parameters based on our integrated network.
Beyond Role-Based Surgical Domain Modeling: Generalizable Re-Identification in the Operating Room
When used to augment markerless personnel tracking, our approach improves accuracy by over 50%.
Skelite: Compact Neural Networks for Efficient Iterative Skeletonization
Skeletonization extracts thin representations from images that compactly encode their geometry and topology.
Rewarding Doubt: A Reinforcement Learning Approach to Confidence Calibration of Large Language Models
A safe and trustworthy use of Large Language Models (LLMs) requires an accurate expression of confidence in their answers.
MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments
Operating rooms (ORs) are complex, high-stakes environments requiring precise understanding of interactions among medical staff, tools, and equipment for enhancing surgical assistance, situational awareness, and patient safety.
Ranked #1 on
Video Panoptic Segmentation
on 4D-OR
(using extra training data)
From Open-Vocabulary to Vocabulary-Free Semantic Segmentation
Open-vocabulary semantic segmentation enables models to identify novel object categories beyond their training data.
Open Vocabulary Semantic Segmentation
Open-Vocabulary Semantic Segmentation
+1
From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine
The field has advanced rapidly, evolving from text-only large language models for tasks such as clinical documentation and decision support to multimodal AI systems capable of integrating diverse data modalities, including imaging, text, and structured data, within a single model.
GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimation
In this paper, we present GCE-Pose, a method that enhances pose estimation for novel instances by integrating category-level global context prior.
Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment
Experiments on the IU X-RAY and MIMIC-CXR radiology datasets demonstrate that Adversarial Domain Alignment enables attackers to steal the medical MLLM without any access to medical data.
EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery
Recently, Multimodal Large Language Models (MLLMs) have demonstrated their immense potential in computer-aided diagnosis and decision-making.
Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis
Conclusion: We propose a text-driven adaptation approach that mitigates the modality gap and handles multiple downstream tasks in surgical workflow analysis, with minimal reliance on large annotated datasets.
UltraRay: Full-Path Ray Tracing for Enhancing Realism in Ultrasound Simulation
We propose a novel ultrasound simulation pipeline that utilizes a ray tracing algorithm to generate echo data, tracing each ray from the transducer through the scene and back to the sensor.
Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis
Scaling by training on large datasets has been shown to enhance the quality and fidelity of image generation and manipulation with diffusion models; however, such large datasets are not always accessible in medical imaging due to cost and privacy issues, which contradicts one of the main applications of such models to produce synthetic samples where real data is scarce.
Conformable Convolution for Topologically Aware Learning of Complex Anatomical Structures
While conventional computer vision emphasizes pixel-level and feature-based objectives, medical image analysis of intricate biological structures necessitates explicit representation of their complex topological properties.
SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians
3D Gaussian Splatting has recently gained traction for its efficient training and real-time rendering.
ESCAPE: Equivariant Shape Completion via Anchor Point Encoding
Shape completion, a crucial task in 3D computer vision, involves predicting and filling the missing regions of scanned or partially observed objects.
G2SDF: Surface Reconstruction from Explicit Gaussians with Implicit SDFs
By establishing a differentiable connection between the explicit Gaussians and the implicit SDF, our approach enables high-quality surface reconstruction and rendering.
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Surgical practice involves complex visual interpretation, procedural skills, and advanced medical knowledge, making surgical vision-language pretraining (VLP) particularly challenging due to this complexity and the limited availability of annotated data.
Synomaly Noise and Multi-Stage Diffusion: A Novel Approach for Unsupervised Anomaly Detection in Ultrasound Imaging
To address these issues, we propose a novel unsupervised anomaly detection framework based on a diffusion model that incorporates a synthetic anomaly (Synomaly) noise function and a multi-stage diffusion process.
SCRREAM : SCan, Register, REnder And Map:A Framework for Annotating Accurate and Dense 3D Indoor Scenes with a Benchmark
However, using these datasets to evaluate dense geometry tasks, such as depth rendering, can be problematic as the meshes of the dataset are often incomplete and may produce wrong ground truth to evaluate the details.
Extrapolating Prospective Glaucoma Fundus Images through Diffusion Model in Irregular Longitudinal Sequences
Subsequently, a time-aligned mask is employed to select a specific year for image generation.
KaLDeX: Kalman Filter based Linear Deformable Cross Attention for Retina Vessel Segmentation
Methods: To address these issues, we propose a novel network (KaLDeX) for vascular segmentation leveraging a Kalman filter based linear deformable cross attention (LDCA) module, integrated within a UNet++ framework.
VISAGE: Video Synthesis using Action Graphs for Surgery
The results of our experiments demonstrate high-fidelity video generation for laparoscopy procedures, which enables various applications in SDS.
Neural Semantic Map-Learning for Autonomous Vehicles
Autonomous vehicles demand detailed maps to maneuver reliably through traffic, which need to be kept up-to-date to ensure a safe operation.
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation
Surgical video-language pretraining (VLP) faces unique challenges due to the knowledge domain gap and the scarcity of multi-modal data.
Physics-Informed Latent Diffusion for Multimodal Brain MRI Synthesis
Our approach utilizes latent diffusion models and a two-step generative process: first, unobserved physical tissue property maps are synthesized using a latent diffusion model, and then these maps are combined with a physical signal model to generate the final MRI scan.
KLDD: Kalman Filter based Linear Deformable Diffusion Model in Retinal Image Segmentation
To better optimize the coordinate positions of deformable convolution, we employ the Kalman filter to enhance the perception of vascular structures in linear deformable convolution.
Intraoperative Registration by Cross-Modal Inverse Neural Rendering
We present in this paper a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering.
SURGIVID: Annotation-Efficient Surgical Video Object Discovery
Considering the profusion of surgical videos obtained through standardized surgical workflows, we propose an annotation-efficient framework for the semantic segmentation of surgical scenes.
MAGDA: Multi-agent guideline-driven diagnostic assistance
Large Language Models (LLMs) have the potential to alleviate some pressure from these clinicians by providing insights that can help them in their decision-making.
Multimodal Analysis of White Blood Cell Differentiation in Acute Myeloid Leukemia Patients using a β-Variational Autoencoder
Biomedical imaging and RNA sequencing with single-cell resolution improves our understanding of white blood cell diseases like leukemia.
PHOCUS: Physics-Based Deconvolution for Ultrasound Resolution Enhancement
Conventionally, deconvolution techniques attempt to rectify the imaging system's dependent PSF, working directly on the radio-frequency (RF) data.
Deep Spectral Methods for Unsupervised Ultrasound Image Interpretation
Ultrasound imaging is challenging to interpret due to non-uniform intensities, low contrast, and inherent artifacts, necessitating extensive training for non-specialists.
Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder
Counterfactual explanations (CEs) aim to enhance the interpretability of machine learning models by illustrating how alterations in input features would affect the resulting predictions.
SANGRIA: Surgical Video Scene Graph Optimization for Surgical Workflow Prediction
Graph-based holistic scene representations facilitate surgical workflow understanding and have recently demonstrated significant success.
SLoRD: Structural Low-Rank Descriptors for Shape Consistency in Vertebrae Segmentation
Specifically, a contour generation network is proposed based on Structural Low-Rank Descriptors for shape consistency, termed SLoRD.
Diffusion as Sound Propagation: Physics-inspired Model for Ultrasound Image Generation
However, when it comes to ultrasound (US) imaging, the authenticity of generated data often diminishes due to the oversight of ultrasound physics.
RaNeuS: Ray-adaptive Neural Surface Reconstruction
Our objective is to leverage a differentiable radiance field \eg NeRF to reconstruct detailed 3D surfaces in addition to producing the standard novel view renderings.
Class-Aware Cartilage Segmentation for Autonomous US-CT Registration in Robotic Intercostal Ultrasound Imaging
Then, a dense skeleton graph-based non-rigid registration is presented to map the intercostal scanning path from a generic template to individual patients.
Ultrasound Report Generation with Cross-Modality Feature Alignment via Unsupervised Guidance
In this work, we propose a novel framework for automatic ultrasound report generation, leveraging a combination of unsupervised and supervised learning methods to aid the report generation process.
HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognition
By disentangling embedding spaces of different hierarchical levels, the learned multi-modal representations encode short-term and long-term surgical concepts in the same model.
MRSegmentator: Multi-Modality Segmentation of 40 Classes in MRI and CT
A human-in-the-loop annotation workflow was employed, leveraging cross-modality transfer learning from an existing CT segmentation model to segment 40 anatomical structures.
EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion
The scheme ensures that the denoising processes are influenced by a holistic understanding of the scene graph, facilitating the generation of globally coherent scenes.
SpecstatOR: Speckle statistics-based iOCT Segmentation Network for Ophthalmic Surgery
This paper presents an innovative approach to intraoperative Optical Coherence Tomography (iOCT) image segmentation in ophthalmic surgery, leveraging statistical analysis of speckle patterns to incorporate statistical pathology-specific prior knowledge.
Autonomous Path Planning for Intercostal Robotic Ultrasound Imaging Using Reinforcement Learning
Ultrasound (US) has been widely used in daily clinical practice for screening internal organs and guiding interventions.
Real-time guidewire tracking and segmentation in intraoperative x-ray
For this purpose, real-time and accurate guidewire segmentation and tracking can enhance the visualization of guidewires and provide visual feedback for physicians during the intervention as well as for robot-assisted interventions.
Shape Completion in the Dark: Completing Vertebrae Morphology from 3D Ultrasound
Purpose: Ultrasound (US) imaging, while advantageous for its radiation-free nature, is challenging to interpret due to only partially visible organs and a lack of complete 3D information.
ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling
This demonstrates ORacle's potential to significantly enhance the scalability and affordability of OR domain modeling and opens a pathway for future advancements in surgical data science.
Ranked #1 on
Scene Graph Generation
on 4D-OR
Neural Cellular Automata for Lightweight, Robust and Explainable Classification of White Blood Cell Images
We test our approach on three datasets of white blood cell images and show that we achieve competitive performance compared to conventional methods.
CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers
In minimally invasive endovascular procedures, contrast-enhanced angiography remains the most robust imaging technique.
VibNet: Vibration-Boosted Needle Detection in Ultrasound Images
The results obtained on porcine samples demonstrate that VibNet effectively detects needles even when their visibility is severely reduced, with a tip error of $1. 61\pm1. 56~mm$ compared to $8. 15\pm9. 98~mm$ for UNet and $6. 63\pm7. 58~mm$ for WNet, and a needle direction error of $1. 64\pm1. 86^{\circ}$ compared to $9. 29\pm15. 30^{\circ}$ for UNet and $8. 54\pm17. 92^{\circ}$ for WNet.
FLex: Joint Pose and Dynamic Radiance Fields Optimization for Stereo Endoscopic Videos
However, the setup has been restricted to a static endoscope, limited deformation, or required an external tracking device to retrieve camera pose information of the endoscopic camera.
MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images
MatchU is a generic approach that fuses 2D texture and 3D geometric cues for 6D pose prediction of unseen objects.
Physics-Encoded Graph Neural Networks for Deformation Prediction under Contact
We also incorporate cross-attention mechanisms to capture the interplay between the objects.
Robot-Assisted Deep Venous Thrombosis Ultrasound Examination using Virtual Fixture
To ensure the biometric measurements obtained in different examinations are comparable, the 6D scanning path is determined in a coarse-to-fine manner using both an external RGBD camera and US images.
On Discprecncies between Perturbation Evaluations of Graph Neural Network Attributions
In this work, we assess attribution methods from a perspective not previously explored in the graph domain: retraining.
HouseCat6D - A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios
Estimating 6D object poses is a major challenge in 3D computer vision.
Deformable 3D Gaussian Splatting for Animatable Human Avatars
ParDy-Human introduces parameter-driven dynamics into 3D Gaussian Splatting where 3D Gaussians are deformed by a human pose model to animate the avatar.
Advancing Surgical VQA with Scene Graph Knowledge
We highlight that the primary limitation in the current surgical VQA systems is the lack of scene knowledge to answer complex queries.
Hybrid Functional Maps for Crease-Aware Non-Isometric Shape Matching
Non-isometric shape correspondence remains a fundamental challenge in computer vision.
Re-Nerfing: Improving Novel View Synthesis through Novel View Synthesis
This introduces additional multi-view constraints and allows the second model to converge to a better solution.
S2P3: Self-Supervised Polarimetric Pose Prediction
The novel training paradigm comprises 1) a physical model to extract geometric information of polarized light, 2) a teacher-student knowledge distillation scheme and 3) a self-supervised loss formulation through differentiable rendering and an invertible physical constraint.
RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance
Conversational AI tools that can generate and discuss clinically correct radiology reports for a given medical image have the potential to transform radiology.
DNS SLAM: Dense Neural Semantic-Informed SLAM
In this work, we introduce DNS SLAM, a novel neural RGB-D semantic SLAM approach featuring a hybrid representation.
Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks
Despite the limited dataset, the GNN-based model significantly outperforms context-agnostic approaches, accurately distinguishing between healthy and tumor tissues, even in images from previously unseen patients.
SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation
Leveraging the advantage of DINOv2 in providing SE(3)-consistent semantic features, we hierarchically extract two types of SE(3)-invariant geometric features to further encapsulate local-to-global object-specific information.
EyeLS: Shadow-Guided Instrument Landing System for Intraocular Target Approaching in Robotic Eye Surgery
Robotic ophthalmic surgery is an emerging technology to facilitate high-precision interventions such as retina penetration in subretinal injection and removal of floating tissues in retinal detachment depending on the input imaging modalities such as microscopy and intraoperative OCT (iOCT).
VoxNeRF: Bridging Voxel Representation and Neural Radiance Fields for Enhanced Indoor View Synthesis
The generation of high-fidelity view synthesis is essential for robotic navigation and interaction but remains challenging, particularly in indoor environments and real-time scenarios.
PRISM: Progressive Restoration for Scene Graph-based Image Manipulation
We take advantage of the outer part of the masked area as they have a direct correlation with the context of the scene.
Dynamic Scene Graph Representation for Surgical Video
Surgical videos captured from microscopic or endoscopic imaging devices are rich but complex sources of information, depicting different tools and anatomical structures utilized during an extended amount of time.
AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers
To date, endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature.
SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs
In this paper, we present SG-Bot, a novel rearrangement framework that utilizes a coarse-to-fine scheme with a scene graph as the scene representation.
RIDE: Self-Supervised Learning of Rotation-Equivariant Keypoint Detection and Invariant Description for Endoscopy
While most classical methods achieve rotation-equivariant detection and invariant description by design, many learning-based approaches learn to be robust only up to a certain degree.
DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields
To address this challenge, we propose Dynamic Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields (DynaMoN).
Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstruction
In this work, we propose the first precise hand-object reconstruction method in hyperbolic space, namely Dynamic Hyperbolic Attention Network (DHANet), which leverages intrinsic properties of hyperbolic space to learn representative features.
BigFUSE: Global Context-Aware Image Fusion in Dual-View Light-Sheet Fluorescence Microscopy with Image Formation Prior
Light-sheet fluorescence microscopy (LSFM), a planar illumination technique that enables high-resolution imaging of samples, experiences defocused image quality caused by light scattering when photons propagate through thick tissues.
On the Localization of Ultrasound Image Slices within Point Distribution Models
We demonstrate that our multi-modal registration framework can localize images on the 3D surface topology of a patient-specific organ and the mean shape of an SSM.
3D Adversarial Augmentations for Robust Out-of-Domain Predictions
We conduct extensive experiments across a variety of scenarios on data from KITTI, Waymo, and CrashD for 3D object detection, and on data from SemanticKITTI, Waymo, and nuScenes for 3D semantic segmentation.
A Continual Learning Approach for Cross-Domain White Blood Cell Classification
Here, we propose a rehearsal-based continual learning approach for class incremental and domain incremental scenarios in white blood cell classification.
A Study of Age and Sex Bias in Multiple Instance Learning based Classification of Acute Myeloid Leukemia Subtypes
To that end, we train multiple MIL models using different levels of sex imbalance in the training set and excluding certain age groups.
Multi-Modal Dataset Acquisition for Photometrically Challenging Object
This paper addresses the limitations of current datasets for 3D vision tasks in terms of accuracy, size, realism, and suitable imaging modalities for photometrically challenging objects.
Polarimetric Information for Multi-Modal 6D Pose Estimation of Photometrically Challenging Objects with Limited Data
6D pose estimation pipelines that rely on RGB-only or RGB-D data show limitations for photometrically challenging objects with e. g. textureless surfaces, reflections or transparency.
Can ultrasound confidence maps predict sonographers' labeling variability?
Therefore, we propose to give the confidence maps as additional information to the networks.
Robust Monocular Depth Estimation under Challenging Conditions
While state-of-the-art monocular depth estimation approaches achieve impressive results in ideal settings, they are highly unreliable under challenging illumination and weather conditions, such as at nighttime or in the presence of rain.
DISBELIEVE: Distance Between Client Models is Very Essential for Effective Local Model Poisoning Attacks
Leveraging this, we introduce DISBELIEVE, a local model poisoning attack that creates malicious parameters or gradients such that their distance to benign clients' parameters or gradients is low respectively but at the same time their adverse effect on the global model's performance is high.
WarpEM: Dynamic Time Warping for Accurate Catheter Registration in EM-guided Procedures
Accurate catheter tracking is crucial during minimally invasive endovascular procedures (MIEP), and electromagnetic (EM) tracking is a widely used technology that serves this purpose.
DefCor-Net: Physics-Aware Ultrasound Deformation Correction
The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis.
LOTUS: Learning to Optimize Task-based US representations
Anatomical segmentation of organs in ultrasound images is essential to many clinical applications, particularly for diagnosis and monitoring.
Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures
We then present a novel method, SurgVLP - Surgical Vision Language Pre-training, for multi-modal representation learning.
DisguisOR: Holistic Face Anonymization for the Operating Room
Methods: RGB and depth images from multiple cameras are fused into a 3D point cloud representation of the scene.
DISA: DIfferentiable Similarity Approximation for Universal Multimodal Registration
Our method is several orders of magnitude faster than local patch-based metrics and can be directly applied in clinical settings by replacing the similarity measure with the proposed one.
Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting
However, there is limited research on automating structured reporting, and no public benchmark is available for evaluating and comparing different methods.
Ranked #1 on
Structured Report Generation
on Rad-ReStruct
Motion Magnification in Robotic Sonography: Enabling Pulsation-Aware Artery Segmentation
To validate the proposed robotic US system for imaging arteries, experiments are carried out on volunteers' carotid and radial arteries.
Thoracic Cartilage Ultrasound-CT Registration using Dense Skeleton Graph
To address this challenge, a graph-based non-rigid registration is proposed to enable transferring planned paths from the atlas to the current setup by explicitly considering subcutaneous bone surface features instead of the skin surface.
Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations
The results demonstrated that the proposed advanced framework can robustly work on a variety of seen and unseen phantoms as well as in-vivo human carotid data.
AutoPaint: A Self-Inpainting Method for Unsupervised Anomaly Detection
This work intends to, first, propose a robust inpainting model to learn the details of healthy anatomies and reconstruct high-resolution images by preserving anatomical constraints.
Self-Supervised Learning for Physiologically-Based Pharmacokinetic Modeling in Dynamic PET
Dynamic positron emission tomography imaging (dPET) provides temporally resolved images of a tracer enabling a quantitative measure of physiological processes.
DopUS-Net: Quality-Aware Robotic Ultrasound Imaging based on Doppler Signal
The experimental results demonstrate that the proposed approach with the re-identification process can significantly improve the accuracy and robustness of the segmentation results (dice score: from 0:54 to 0:86; intersection over union: from 0:47 to 0:78).
Skeleton Graph-based Ultrasound-CT Non-rigid Registration
Autonomous ultrasound (US) scanning has attracted increased attention, and it has been seen as a potential solution to overcome the limitations of conventional US examinations, such as inter-operator variations.
Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments
Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments and analyze the influence of camera configurations, training data, and occlusions on the pose accuracy and generalization ability.
Incremental 3D Semantic Scene Graph Prediction from RGB Sequences
Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network.
SCOPE: Structural Continuity Preservation for Medical Image Segmentation
Although the preservation of shape continuity and physiological anatomy is a natural assumption in the segmentation of medical images, it is often neglected by deep learning methods that mostly aim for the statistical modeling of input data as pixels rather than interconnected structures.
DIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation
Although purely transformer-based architectures showed promising performance in many computer vision tasks, many hybrid models consisting of CNN and transformer blocks are introduced to fit more specialized tasks.
SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis
To address this limitation, we propose a novel guidance approach for the sampling process in the diffusion model that leverages bounding box and segmentation map information at inference time without additional training data.
S3M: Scalable Statistical Shape Modeling through Unsupervised Correspondences
Statistical shape models (SSMs) are an established way to represent the anatomy of a population with various clinically relevant applications.
Whether and When does Endoscopy Domain Pretraining Make Sense?
In this work, we investigate the need for endoscopy domain-specific pretraining based on downstream objectives.
On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks
Learning-based methods to solve dense 3D vision problems typically train on 3D sensor data.
Prior-RadGraphFormer: A Prior-Knowledge-Enhanced Transformer for Generating Radiology Graphs from X-Rays
The extraction of structured clinical information from free-text radiology reports in the form of radiology graphs has been demonstrated to be a valuable approach for evaluating the clinical correctness of report-generation methods.
Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis
Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making.
LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Rooms
The holistic representation of surgical scenes as semantic scene graphs (SGG), where entities are represented as nodes and relations between them as edges, is a promising direction for fine-grained semantic OR understanding.
Ranked #4 on
Scene Graph Generation
on 4D-OR
MI-SegNet: Mutual Information-Based US Segmentation for Unseen Domain Generalization
We validate the generalizability of the proposed domain-independent segmentation approach on several datasets with varying parameters and machines.
Semantic Latent Space Regression of Diffusion Autoencoders for Vertebral Fracture Grading
We then regress the severity of the fracture as a function of the distance to this hyperplane, calibrating the results to the Genant scale.
Location-Free Scene Graph Generation
Scene Graph Generation (SGG) is a visual understanding task, aiming to describe a scene as a graph of entities and their relationships with each other.
Ranked #3 on
Scene Graph Generation
on 4D-OR
Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs
Image synthesis driven by computer graphics achieved recently a remarkable realism, yet synthetic image data generated this way reveals a significant domain gap with respect to real-world data.
Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Images
Explainability is a key requirement for computer-aided diagnosis systems in clinical decision-making.
Active Learning Enhances Classification of Histopathology Whole Slide Images with Attention-based Multiple Instance Learning
Querying the expert to annotate regions of interest in a WSI guides the formation of high-attention regions for MIL.
BEL: A Bag Embedding Loss for Transformer enhances Multiple Instance Whole Slide Image Classification
Recent MIL approaches produce highly informative bag level representations by utilizing the transformer architecture's ability to model the dependencies between instances.
IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction
Reliable multi-agent trajectory prediction is crucial for the safe planning and control of autonomous systems.
CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection.
Ranked #1 on
Action Triplet Detection
on CholecT50 (Challenge)
Investigating Pulse-Echo Sound Speed Estimation in Breast Ultrasound with Deep Learning
However, the various types of breast tissue, such as glandular, fat, and lesions, differ in sound speed.
KST-Mixer: Kinematic Spatio-Temporal Data Mixer For Colon Shape Estimation
Kinematic data of a colonoscope and the colon, including positions and directions of their centerlines, are obtained using electromagnetic and depth sensors.
Lidar Upsampling with Sliced Wasserstein Distance
This in turn enables our method to employ a one-stage upsampling paradigm without the need for coarse and fine reconstruction.
Ultra-NeRF: Neural Radiance Fields for Ultrasound Imaging
In our work, we discuss direction-dependent changes in the scene and show that a physics-inspired rendering improves the fidelity of US image synthesis.
Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing
In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes.
TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimation
In this paper, we introduce neural texture learning for 6D object pose estimation from synthetic data and a few unlabelled real images.
SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environments
We introduce a zero-shot split for Tabletop Objects Dataset (TOD-Z) to enable this study and present a method that uses annotated objects to learn the ``objectness'' of pixels and generalize to unseen object categories in cluttered indoor environments.
HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios
Estimating 6D object poses is a major challenge in 3D computer vision.
DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation
Graph representation of objects and their relations in a scene, known as a scene graph, provides a precise and discernible interface to manipulate a scene by modifying the nodes or the edges in the graph.
Improved Techniques for the Conditional Generative Augmentation of Clinical Audio Data
In contrast to previously proposed fully convolutional models, the proposed model implements residual Squeeze and Excitation modules in the generator architecture.
What can we learn about a generated image corrupting its latent representation?
The purpose of this work is to investigate the hypothesis that we can predict image quality based on its latent representation in the GANs bottleneck.
MonoGraspNet: 6-DoF Grasping with a Single RGB Image
6-DoF robotic grasping is a long-lasting but unsolved problem.
Segmenting Known Objects and Unseen Unknowns without Prior Knowledge
By doing so, for the first time in panoptic segmentation with unknown objects, our U3HS is trained without unknown categories, reducing assumptions and leaving the settings as unconstrained as in real-life scenarios.
Ranked #3 on
Object Detection
on OoDIS
Towards Autonomous Atlas-based Ultrasound Acquisitions in Presence of Articulated Motion
Robotic ultrasound (US) imaging aims at overcoming some of the limitations of free-hand US examinations, e. g. difficulty in guaranteeing intra- and inter-operator repeatability.
DA$^2$ Dataset: Toward Dexterity-Aware Dual-Arm Grasping
In this paper, we introduce DA$^2$, the first large-scale dual-arm dexterity-aware dataset for the generation of optimal bimanual grasping pairs for arbitrary large objects.
Speckle2Speckle: Unsupervised Learning of Ultrasound Speckle Filtering Without Clean Data
To enable this, we make use of realistic ultrasound simulation techniques that allow for instantiation of several independent speckle realizations that represent the exact same tissue, thus allowing for the application of image reconstruction techniques that work with pairs of differently corrupted data.
CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning
The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods while requiring significantly fewer computations.
Spotlight on nerves: Portable multispectral optoacoustic imaging of peripheral nerve vascularization and morphology
Various morphological and functional parameters of peripheral nerves and their vascular supply are indicative of pathological changes due to injury or disease.
Exploiting Diversity of Unlabeled Data for Label-Efficient Semi-Supervised Active Learning
By considering the consistency information with the diversity in the consistency-based embedding scheme, the proposed method could select more informative samples for labeling in the semi-supervised learning setting.
Unsupervised pre-training of graph transformers on patient population graphs
We find that our proposed pre-training methods help in modeling the data at a patient and population level and improve performance in different fine-tuning tasks on all datasets.
CACTUSS: Common Anatomical CT-US Space for US examinations
Abdominal aortic aneurysm (AAA) is a vascular disease in which a section of the aorta enlarges, weakening its walls and potentially rupturing the vessel.
CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN
Deep learning models used in medical image analysis are prone to raising reliability concerns due to their black-box nature.
Shape-Aware Masking for Inpainting in Medical Imaging
Inpainting has recently been proposed as a successful deep learning technique for unsupervised medical image model discovery.
Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data
Although most medical centers conduct similar medical imaging tasks, their differences, such as specializations, number of patients, and devices, lead to distinctive data distributions.
Unsupervised Cross-Domain Feature Extraction for Single Blood Cell Image Classification
Here, we propose a cross-domain adapted autoencoder to extract features in an unsupervised manner on three different datasets of single white blood cells scanned from peripheral blood smears.
DeStripe: A Self2Self Spatio-Spectral Graph Neural Network with Unfolded Hessian for Stripe Artifact Removal in Light-sheet Microscopy
Light-sheet fluorescence microscopy (LSFM) is a cutting-edge volumetric imaging technique that allows for three-dimensional imaging of mesoscopic samples with decoupled illumination and detection paths.
U-PET: MRI-based Dementia Detection with Joint Generation of Synthetic FDG-PET Images
To this end, we propose a multi-task method based on U-Net that takes T1-weighted MR images as an input to generate synthetic FDG-PET images and classifies the dementia progression of the patient into cognitive normal (CN), cognitive impairment (MCI), and AD.
PRO-TIP: Phantom for RObust automatic ultrasound calibration by TIP detection
We propose a novel method to automatically calibrate tracked ultrasound probes.
Virtual embeddings and self-consistency for self-supervised learning
We validate TriMix on eight benchmark datasets consisting of natural and medical images with an improvement of 2. 71% and 0. 41% better than the second-best models for both data types.
BFS-Net: Weakly Supervised Cell Instance Segmentation from Bright-Field Microscopy Z-Stacks
Despite its broad availability, volumetric information acquisition from Bright-Field Microscopy (BFM) is inherently difficult due to the projective nature of the acquisition process.
PhoCaL: A Multi-Modal Dataset for Category-Level Object Pose Estimation with Photometrically Challenging Objects
Object pose estimation is crucial for robotic applications and augmented reality.
Weakly-supervised Biomechanically-constrained CT/MRI Registration of the Spine
We specifically design these losses to depend only on the CT label maps since automatic vertebra segmentation in CT gives more accurate results contrary to MRI.
VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation
The results demonstrate that proposed approach can effectively and accurately navigate the probe towards the longitudinal view of vessels.
Affective Medical Estimation and Decision Making via Visualized Learning and Deep Learning
With the advent of sophisticated machine learning (ML) techniques and the promising results they yield, especially in medical applications, where they have been investigated for different tasks to enhance the decision-making process.
SoftPool++: An Encoder-Decoder Network for Point Cloud Completion
We propose a novel convolutional operator for the task of point cloud completion.
Y-Net: A Spatiospectral Dual-Encoder Networkfor Medical Image Segmentation
Automated segmentation of retinal optical coherence tomography (OCT) images has become an important recent direction in machine learning for medical applications.
Ranked #1 on
Retinal OCT Layer Segmentation
on Duke SD-OCT
(using extra training data)
Analyzing the Effects of Handling Data Imbalance on Learned Features from Medical Images by Looking Into the Models
One challenging property lurking in medical datasets is the imbalanced data distribution, where the frequency of the samples between the different classes is not balanced.
Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications
In this work, we propose Graph-in-Graph (GiG), a neural network architecture for protein classification and brain imaging applications that exploits the graph representation of the input data samples and their latent relation.
Learning Local Displacements for Point Cloud Completion
To this aim, we introduce a second model that assembles our layers within a transformer architecture.
Interpretable Vertebral Fracture Diagnosis
Do black-box neural network models learn clinically relevant features for fracture diagnosis?
FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays
The automation of chest X-ray reporting has garnered significant interest due to the time-consuming nature of the task.
Intelligent Masking: Deep Q-Learning for Context Encoding in Medical Image Analysis
We show that training the agent against the prediction model can significantly improve the semantic features extracted for downstream classification tasks.
Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions
We test our method on two medical datasets of patient records, TADPOLE and MIMIC-III, including imaging and non-imaging features and different prediction tasks.
Ranked #1 on
Length-of-Stay prediction
on MIMIC-III
4D-OR: Semantic Scene Graphs for OR Domain Modeling
Towards this goal, for the first time, we propose using semantic scene graphs (SSG) to describe and summarize the surgical scene.
Ranked #5 on
Scene Graph Generation
on 4D-OR
Conditional Generative Data Augmentation for Clinical Audio Datasets
In this work, we propose a novel data augmentation method for clinical audio datasets based on a conditional Wasserstein Generative Adversarial Network with Gradient Penalty (cWGAN-GP), operating on log-mel spectrograms.
Longitudinal Self-Supervision for COVID-19 Pathology Quantification
For this purpose, longitudinal self-supervision schemes are explored on clinical longitudinal COVID-19 CT scans.
ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation
Dense methods also improved pose estimation in the presence of occlusion.
Surgical Workflow Recognition: from Analysis of Challenges to Architectural Study
Algorithmic surgical workflow recognition is an ongoing research field and can be divided into laparoscopic (Internal) and operating room (External) analysis.
Know your sensORs -- A Modality Study For Surgical Action Classification
Existing datasets from OR room cameras are thus far limited in size or modalities acquired, leaving it unclear which sensor modalities are best suited for tasks such as recognizing surgical action from videos.
From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Prediction
There have been numerous recently proposed methods for monocular depth prediction (MDP) coupled with the equally rapid evolution of benchmarking tools.
GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting
While 6D object pose estimation has recently made a huge leap forward, most methods can still only handle a single or a handful of different objects, which limits their applications.
Ranked #1 on
6D Pose Estimation
on LineMOD
(Mean ADD-S metric)
Do Explanations Explain? Model Knows Best
It is a mystery which input features contribute to a neural network's output.
Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transport
Shape matching has been a long-studied problem for the computer graphics and vision community.
Transformers in Action: Weakly Supervised Action Segmentation
The video action segmentation task is regularly explored under weaker forms of supervision, such as transcript supervision, where a list of actions is easier to obtain than dense frame-wise labels.
A Variational Bayesian Method for Similarity Learning in Non-Rigid Image Registration
We propose a novel variational Bayesian formulation for diffeomorphic non-rigid registration of medical images, which learns in an unsupervised way a data-specific similarity metric.
3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection
Despite training only on a standard dataset, such as KITTI, augmenting with our vector fields significantly improves the generalization to differently shaped objects and scenes.
Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments
Indirect Time-of-Flight (I-ToF) imaging is a widespread way of depth estimation for mobile devices due to its small size and affordable price.
DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration
We first present a small sequence of RGB-D images displaying a human-object interaction.
Object-aware Monocular Depth Prediction with Instance Convolutions
With the advent of deep learning, estimating depth from a single RGB image has recently received a lot of attention, being capable of empowering many different applications ranging from path planning for robotics to computational cinematography.
ColibriDoc: An Eye-in-Hand Autonomous Trocar Docking System
For this purpose, we present a platform for autonomous trocar docking that combines computer vision and a robotic setup.
MIGS: Meta Image Generation from Scene Graphs
We propose MIGS (Meta Image Generation from Scene Graphs), a meta-learning based approach for few-shot image generation from graphs that enables adapting the model to different scenes and increases the image quality by training on diverse sets of tasks.
Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation
A novel temporal attention mechanism further processes the local geometric information in a global context across consecutive images.
Semantic Image Alignment for Vehicle Localization
Accurate and reliable localization is a fundamental requirement for autonomous vehicles to use map information in higher-level tasks such as navigation or planning.
CertainNet: Sampling-free Uncertainty Estimation for Object Detection
Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings.
Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information
We propose a method to identify features with predictive information in the input domain.
Interactive Segmentation for COVID-19 Infection Quantification on Longitudinal CT scans
Existing automatic and interactive segmentation models for medical images only use data from a single time point (static).
Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation
The results of our experiments show that the proposed method improves the network's performance on real images by a considerable margin and can be employed in 3D reconstruction pipelines.
MetaMedSeg: Volumetric Meta-learning for Few-Shot Organ Segmentation
In this work, we present MetaMedSeg, a gradient-based meta-learning algorithm that redefines the meta-learning task for the volumetric medical data with the goal to capture the variety between the slices.
Sickle Cell Disease Severity Prediction from Percoll Gradient Images using Graph Convolutional Networks
Sickle cell disease (SCD) is a severe genetic hemoglobin disorder that results in premature destruction of red blood cells.
Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs
Scene graphs are representations of a scene, composed of objects (nodes) and inter-object relationships (edges), proven to be particularly suited for this task, as they allow for semantic control on the generated content.
SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation
Directly regressing all 6 degrees-of-freedom (6DoF) for the object pose (e. g. the 3D rotation and translation) in a cluttered environment from a single RGB image is a challenging problem.
Ranked #1 on
6D Pose Estimation using RGB
on Occlusion LineMOD
Unconditional Scene Graph Generation
Scene graphs, composed of nodes as objects and directed-edges as relationships among objects, offer an alternative representation of a scene that is more semantically grounded than images.
Tracked 3D Ultrasound and Deep Neural Network-based Thyroid Segmentation reduce Interobserver Variability in Thyroid Volumetry
Conclusion: Tracked 3D ultrasound combined with a CNN segmentation significantly reduces interobserver variability in thyroid volumetry and increases the accuracy of the measurements with shorter acquisition times.
R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes
While self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches, violations of the static world assumption can still lead to erroneous depth predictions of traffic participants, posing a potential safety issue.
U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction
Specifically, we introduce a multimodal similarity metric to build a population graph for clustering patients and an image-based end-to-end Graph Attention Network to process this graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation and mortality.
Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP
Derived regions are consistent across different images and coincide with human-defined semantic classes on some datasets.
Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images
In this work, we introduce Deep Direct Volume Rendering (DeepDVR), a generalization of DVR that allows for the integration of deep neural networks into the DVR algorithm.
Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedures
We then use MSSG to introduce a dynamically generated graphical user interface tool for surgical procedure analysis which could be used for many applications including process optimization, OR design and automatic report generation.
