Search Results for author:
Found 64 papers, 45 papers with code
Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglement
We propose a multimodal Graph Representation network with Adversarial feature Disentanglement (GRAD) for robust surgical workflow recognition in challenging scenarios with domain shifts or corrupted data.
Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Cameras
Additionally, we propose a 3D scene reconstruction pipeline that optimizes depth maps' scales, shifts, and a few parameters based on our integrated network.
SurgicalVLM-Agent: Towards an Interactive AI Co-Pilot for Pituitary Surgery
Image-guided surgery demands adaptive, real-time decision support, yet static AI models struggle with structured task planning and providing interactive guidance.
Endo-FASt3r: Endoscopic Foundation model Adaptation for Structure from motion
Despite recent advancements in foundation model adaptation to monocular depth estimation of endoscopic scenes via self-supervised learning (SSL), no prior work has explored their use for pose estimation.
PitVQA++: Vector Matrix-Low-Rank Adaptation for Open-Ended Visual Question Answering in Pituitary Surgery
However, the development of VLMs for surgical VQA is challenging due to limited datasets and the risk of overfitting and catastrophic forgetting during full fine-tuning of pretrained weights.
Transferring Knowledge from High-Quality to Low-Quality MRI for Adult Glioma Diagnosis
Our top-performing model achieves Dice scores of 0. 882, 0. 840, and 0. 926, and Hausdorff Distance (95%) scores of 15. 324, 37. 518, and 13. 971 for enhancing tumor, tumor core, and whole tumor, respectively, in the validation phase.
SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction
Accurate 3D reconstruction of dynamic surgical scenes from endoscopic video is essential for robotic-assisted surgery.
DARES: Depth Anything in Robotic Endoscopic Surgery with Self-supervised Vector-LoRA of the Foundation Model
To tackle this issue, we introduce Depth Anything in Robotic Endoscopic Surgery (DARES), a novel approach that employs a new adaptation technique, Vector Low-Rank Adaptation (Vector-LoRA) on the DAM V2 to perform self-supervised monocular depth estimation in RAS scenes.
Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery
In particular, surgical VQA can enhance the interpretation of surgical data, aiding in accurate diagnoses, effective education, and clinical interventions.
SAM 2 in Robotic Surgery: An Empirical Evaluation for Robustness and Generalization in Surgical Video Segmentation
The recent Segment Anything Model (SAM) 2 has demonstrated remarkable foundational competence in semantic segmentation, with its memory mechanism and mask decoder further addressing challenges in video tracking and object occlusion, thereby achieving superior results in interactive segmentation for both images and videos.
EndoUIC: Promptable Diffusion Transformer for Unified Illumination Correction in Capsule Endoscopy
While researchers have discussed the challenges of low-light enhancement in WCE, the issue of correcting for different exposure levels remains underexplored.
PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgery
This paper introduces PitVQA, a novel dataset specifically designed for VQA in endonasal pituitary surgery and PitVQA-Net, an adaptation of the GPT2 with a novel image-grounded text embedding for surgical VQA.
Illumination Histogram Consistency Metric for Quantitative Assessment of Video Sequences
In this paper, we propose a illumination histogram consistency (IHC) metric to quantitatively and automatically evaluate the illumination consistency of the video sequences.
EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Camera
We propose Endoscopic Depth Any Camera (EndoDAC) which is an efficient self-supervised depth estimation framework that adapts foundation models to endoscopic scenes.
Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery
We utilise a commonly used detection architecture, DETR, and fine-tuned it to obtain bounding box prompt for the instruments.
Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgery
Recent advancements in Surgical Visual Question Answering (Surgical-VQA) and related region grounding have shown great promise for robotic and medical applications, addressing the critical need for automated methods in personalized surgical mentorship.
LLM-Assisted Multi-Teacher Continual Learning for Visual Question Answering in Robotic Surgery
We further design an adaptive weight assignment approach that balances the generalization ability of the LLM and the domain expertise of the old CL model.
EndoOOD: Uncertainty-aware Out-of-distribution Detection in Capsule Endoscopy Diagnosis
Wireless capsule endoscopy (WCE) is a non-invasive diagnostic procedure that enables visualization of the gastrointestinal (GI) tract.
OSSAR: Towards Open-Set Surgical Activity Recognition in Robot-assisted Surgery
Existing algorithms dedicated to surgical activity recognition predominantly cater to pre-defined closed-set paradigms, ignoring the challenges of real-world open-set scenarios.
Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery
The problem becomes worse when it limits releasing the dataset of the old instruments for the old model due to privacy concerns and the unavailability of the data for the new or updated version of the instruments for the continual learning model.
Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splatting
In the realm of robot-assisted minimally invasive surgery, dynamic scene reconstruction can significantly enhance downstream tasks and improve surgical outcomes.
Surgical-DINO: Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgery
There is clear evidence in the results that zero-shot prediction on pre-trained weights in computer vision datasets or naive fine-tuning is not sufficient to use the foundation model in the surgical domain directly.
SAM Meets Robotic Surgery: An Empirical Study on Generalization, Robustness and Adaptation
Our extensive evaluation results reveal that although SAM shows remarkable zero-shot generalization ability with bounding box prompts, it struggles to segment the whole instrument with point-based prompts and unprompted settings.
Robustness Stress Testing in Medical Image Classification
We conclude that progressive stress testing is a viable and important tool and should become standard practice in the clinical validation of image-based disease detection models.
Landmark Detection using Transformer Toward Robot-assisted Nasal Airway Intubation
This paper proposes a transformer-based landmark detection solution with deformable DeTR and the semantic-aligned-matching module for detecting landmarks in robot-assisted intubation.
Revisiting Distillation for Continual Learning on Visual Question Localized-Answering in Robotic Surgery
We further establish a CL framework on three public surgical datasets in the context of surgical settings that consist of overlapping classes between old and new surgical VQLA tasks.
SimCol3D -- 3D Reconstruction during Colonoscopy Challenge
We show that depth prediction from synthetic colonoscopy images is robustly solvable, while pose estimation remains an open research question.
CAT-ViL: Co-Attention Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery
Therefore, a surgical Visual Question Localized-Answering (VQLA) system would be helpful for medical students and junior surgeons to learn and understand from recorded surgical videos.
LLCaps: Learning to Illuminate Low-Light Capsule Endoscopy with Curved Wavelet Attention and Reverse Diffusion
Given the exuberant development of the denoising diffusion probabilistic model (DDPM) in computer vision, we introduce a WCE LLIE framework based on the multi-scale convolutional neural network (CNN) and reverse diffusion process.
Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis
In this work, we mitigate data-related issues by efficiently leveraging minimal source images to generate synthetic surgical instrument segmentation datasets and achieve outstanding generalization performance on unseen real domains.
Curriculum-Based Augmented Fourier Domain Adaptation for Robust Medical Image Segmentation
Accurate and robust medical image segmentation is fundamental and crucial for enhancing the autonomy of computer-aided diagnosis and intervention systems.
S$^2$ME: Spatial-Spectral Mutual Teaching and Ensemble Learning for Scribble-supervised Polyp Segmentation
Furthermore, to produce reliable mixed pseudo labels, which enhance the effectiveness of ensemble learning, we introduce a novel adaptive pixel-wise fusion technique based on the entropy guidance from the spatial and spectral branches.
Surgical-VQLA: Transformer with Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery
In this paper, we propose Visual Question Localized-Answering in Robotic Surgery (Surgical-VQLA) to localize the specific surgical area during the answer prediction.
Surgical tool classification and localization: results and methods from the MICCAI 2022 SurgToolLoc challenge
Unfortunately, obtaining the annotations needed to train machine learning models to identify and localize surgical tools is a difficult task.
SAM Meets Robotic Surgery: An Empirical Study in Robustness Perspective
In this empirical study, we investigate the robustness and zero-shot generalizability of the SAM in the domain of robotic surgery in various settings of (i) prompted vs. unprompted; (ii) bounding box vs. points-based prompt; (iii) generalization under corruptions and perturbations with five severity levels; and (iv) state-of-the-art supervised model vs. SAM.
SurgicalGPT: End-to-End Language-Vision GPT for Visual Question Answering in Surgery
Given the limitations of unidirectional attention in GPT models and their ability to generate coherent long paragraphs, we carefully sequence the word tokens before vision tokens, mimicking the human thought process of understanding the question to infer an answer from an image.
Paced-Curriculum Distillation with Prediction and Label Uncertainty for Image Segmentation
Purpose: In curriculum learning, the idea is to train on easier samples first and gradually increase the difficulty, while in self-paced learning, a pacing function defines the speed to adapt the training progress.
Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understanding
In this work, we design a paced curriculum by label smoothing (P-CBLS) using paced learning with uniform label smoothing (ULS) for classification tasks and fuse uniform and spatially varying label smoothing (SVLS) for semantic segmentation tasks in a curriculum manner.
Task-Aware Asynchronous Multi-Task Model with Class Incremental Contrastive Learning for Surgical Scene Understanding
Conclusion: The proposed multi-task model was able to adapt to domain shifts, incorporate novel instruments in the target domain, and perform tool-tissue interaction detection and report generation on par with single-task models.
Bi-Link: Bridging Inductive Link Predictions from Text via Contrastive Learning of Transformers and Prompts
In our experiments, Bi-Link outperforms recent baselines on link prediction datasets (WN18RR, FB15K-237, and Wikidata5M).
Frequency Dropout: Feature-Level Regularization via Randomized Filtering
Both high and low frequencies can be characteristic of the underlying noise distribution caused by the image acquisition rather than in relation to the task-relevant information about the image content.
Estimating Model Performance under Domain Shifts with Class-Specific Confidence Scores
If we could estimate the performance that a pre-trained model would achieve on data from a specific deployment setting, for example a certain clinic, we could judge whether the model could safely be deployed or if its performance degrades unacceptably on the specific data.
Angular Gap: Reducing the Uncertainty of Image Difficulty through Model Calibration
In this work, we propose Angular Gap, a measure of difficulty based on the difference in angular distance between feature embeddings and class-weight embeddings built by hyperspherical learning.
Rethinking Surgical Captioning: End-to-End Window-Based MLP Transformer Using Patches
Surgical captioning plays an important role in surgical instruction prediction and report generation.
Rethinking Surgical Instrument Segmentation: A Background Image Can Be All You Need
Our empirical analysis suggests that without the high cost of data collection and annotation, we can achieve decent surgical instrument segmentation performance.
Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformer
This overload often limits their time answering questionnaires from patients, medical students or junior residents related to surgical procedures.
Class Balanced PixelNet for Neurological Image Segmentation
We deal with this problem by selecting an equal number of pixels for all the classes in sampling time.
Ischemic Stroke Lesion Segmentation Using Adversarial Learning
Training a segmentation network along with an adversarial network can detect and correct higher order inconsistencies between the segmentation maps produced by ground-truth and the Segmentor.
CholecTriplet2021: A benchmark challenge for surgical action triplet recognition
In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge.
Ranked #1 on
Action Triplet Recognition
on CholecT50 (Challenge)
(using extra training data)
Global-Reasoned Multi-Task Learning Model for Surgical Scene Understanding
Global and local relational reasoning enable scene understanding models to perform human-like scene analysis and understanding.
ST-MTL: Spatio-Temporal Multitask Learning Model to Predict Scanpath While Tracking Instruments in Robotic Surgery
We generate the task-aware saliency maps and scanpath of the instruments on the dataset of the MICCAI 2017 robotic instrument segmentation challenge.
2020 CATARACTS Semantic Segmentation Challenge
In 2020, we released pixel-wise semantic annotations for anatomy and instruments for 4670 images sampled from 25 videos of the CATARACTS training set.
Class-Distribution-Aware Calibration for Long-Tailed Visual Recognition
In CDA-TS, the scalar temperature value is replaced with the CDA temperature vector encoded with class frequency to compensate for the over-confidence.
Class-Incremental Domain Adaptation with Smoothing and Calibration for Surgical Report Generation
To adapt incremental classes and extract domain invariant features, a class-incremental (CI) learning method with supervised contrastive (SupCon) loss is incorporated with a feature extractor.
Spatially Varying Label Smoothing: Capturing Uncertainty from Expert Annotations
The task of image segmentation is inherently noisy due to ambiguities regarding the exact location of boundaries between anatomical structures.
Brain Tumor Segmentation and Survival Prediction using 3D Attention UNet
For survival prediction, we extract some novel radiomic features based on geometry, location, the shape of the segmented tumor and combine them with clinical information to estimate the survival duration for each patient.
Glioma Prognosis: Segmentation of the Tumor and Survival Prediction using Shape, Geometric and Clinical Information
In this paper, we exploit a convolutional neural network (CNN) with hypercolumn technique to segment tumor from healthy brain tissue.
Glioblastoma Multiforme Prognosis: MRI Missing Modality Generation, Segmentation and Radiogenomic Survival Prediction
The proposed approaches are evaluated by comparative experiments with state-of-the-art models in synthesis, segmentation, and overall survival (OS) prediction.
Radiogenomics of Glioblastoma: Identification of Radiomics associated with Molecular Subtypes
Glioblastoma is the most malignant type of central nervous system tumor with GBM subtypes cleaved based on molecular level gene alterations.
Real-Time Instrument Segmentation in Robotic Surgery using Auxiliary Supervised Deep Adversarial Learning
Real-time semantic segmentation of the robotic instruments and tissues is a crucial step in robot-assisted surgery.
Learning and Reasoning with the Graph Structure Representation in Robotic Surgery
Learning to infer graph representations and performing spatial reasoning in a complex surgical environment can play a vital role in surgical scene understanding in robotic surgery.
AP-MTL: Attention Pruned Multi-task Learning Model for Real-time Instrument Detection and Segmentation in Robot-assisted Surgery
Training a real-time robotic system for the detection and segmentation of high-resolution images provides a challenging problem with the limited computational resource.
Learning Where to Look While Tracking Instruments in Robot-assisted Surgery
For this purpose, we propose an end-to-end trainable multitask learning (MTL) model for real-time surgical instrument segmentation and attention prediction.
Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i. e., 2012-2018.
