Search Results for author:
Found 102 papers, 34 papers with code
NCDD: Nearest Centroid Distance Deficit for Out-Of-Distribution Detection in Gastrointestinal Vision
Evaluations across multiple deep learning architectures and two publicly available benchmarks, Kvasir2 and Gastrovision, demonstrate the effectiveness of our approach compared to several state-of-the-art methods.
RGB to Hyperspectral: Spectral Reconstruction for Enhanced Surgical Imaging
Transformer models exhibit superior performance in terms of RMSE, SAM, PSNR and SSIM by effectively integrating spatial information to predict accurate spectral profiles, encompassing both visible and extended spectral ranges.
Robotic Arm Platform for Multi-View Image Acquisition and 3D Reconstruction in Minimally Invasive Surgery
Our best reconstruction results occur with operating room lighting and spherical trajectories.
SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction
Accurate 3D reconstruction of dynamic surgical scenes from endoscopic video is essential for robotic-assisted surgery.
Tracking Everything in Robotic-Assisted Surgery
However, its efficacy in surgical scenarios remains untested, largely due to the lack of a comprehensive surgical tracking dataset for evaluation.
PitRSDNet: Predicting Intra-operative Remaining Surgery Duration in Endoscopic Pituitary Surgery
In endoscopic pituitary surgery, it is uniquely challenging due to variable workflow sequences with a selection of optional steps contributing to high variability in surgery duration.
Automated Surgical Skill Assessment in Endoscopic Pituitary Surgery using Real-time Instrument Tracking on a High-fidelity Bench-top Phantom
This therefore demonstrates the feasibility of automated surgical skill assessment in simulated endoscopic pituitary surgery.
PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgery
The field of computer vision applied to videos of minimally invasive surgery is ever-growing.
DARES: Depth Anything in Robotic Endoscopic Surgery with Self-supervised Vector-LoRA of the Foundation Model
To tackle this issue, we introduce Depth Anything in Robotic Endoscopic Surgery (DARES), a novel approach that employs a new adaptation technique, Vector Low-Rank Adaptation (Vector-LoRA) on the DAM V2 to perform self-supervised monocular depth estimation in RAS scenes.
Mismatched: Evaluating the Limits of Image Matching Approaches and Benchmarks
Given the rapid pace of research and the frequent introduction of new image matching methods, it is essential to evaluate them.
CathAction: A Benchmark for Endovascular Intervention Understanding
Real-time visual feedback from catheterization analysis is crucial for enhancing surgical safety and efficiency during endovascular interventions.
Personalizing Federated Instrument Segmentation with Visual Trait Priors in Robotic Surgery
To preserve the unique appearance representation of each site and gradually leverage the inter-site difference, APE introduces appearance regulation and provides customized layer-wise aggregation solutions via hypernetworks for each site's personalized parameters.
HUP-3D: A 3D multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimation
We present HUP-3D, a 3D multi-view multi-modal synthetic dataset for hand-ultrasound (US) probe pose estimation in the context of obstetric ultrasound.
Think Step by Step: Chain-of-Gesture Prompting for Error Detection in Robotic Surgical Videos
Our method encapsulates the reasoning processes inherent to surgical activities enabling it to outperform the state-of-the-art by 4. 6% in F1 score, 4. 6% in Accuracy, and 5. 9% in Jaccard index while processing each frame in 6. 69 milliseconds on average, demonstrating the great potential of our approach in enhancing the safety and efficacy of RMIS procedures and surgical education.
SEDMamba: Enhancing Selective State Space Modelling with Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgery
SEDMamba enhances selective SSM with a bottleneck mechanism and fine-to-coarse temporal fusion (FCTF) to detect and temporally localize surgical errors in long videos.
PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgery
This paper introduces PitVQA, a novel dataset specifically designed for VQA in endonasal pituitary surgery and PitVQA-Net, an adaptation of the GPT2 with a novel image-grounded text embedding for surgical VQA.
High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces
In surgical oncology, screening colonoscopy plays a pivotal role in providing diagnostic assistance, such as biopsy, and facilitating surgical navigation, particularly in polyp detection.
Measuring proximity to standard planes during fetal brain ultrasound scanning
This paper introduces a novel pipeline designed to bring ultrasound (US) plane pose estimation closer to clinical use for more effective navigation to the standard planes (SPs) in the fetal brain.
Gaussian Pancakes: Geometrically-Regularized 3D Gaussian Splatting for Realistic Endoscopic Reconstruction
Within colorectal cancer diagnostics, conventional colonoscopy techniques face critical limitations, including a limited field of view and a lack of depth information, which can impede the detection of precancerous lesions.
Novel View Synthesis
Simultaneous Localization and Mapping
+1
SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge
Surgical tool segmentation and action recognition are fundamental building blocks in many computer-assisted intervention applications, ranging from surgical skills assessment to decision support systems.
Federated Active Learning for Target Domain Generalisation
In this paper, we introduce Active Learning framework in Federated Learning for Target Domain Generalisation, harnessing the strength from both learning paradigms.
Task-guided Domain Gap Reduction for Monocular Depth Prediction in Endoscopy
Colorectal cancer remains one of the deadliest cancers in the world.
Latent Disentanglement in Mesh Variational Autoencoders Improves the Diagnosis of Craniofacial Syndromes and Aids Surgical Planning
The use of deep learning to undertake shape analysis of the complexities of the human head holds great promise.
SimCol3D -- 3D Reconstruction during Colonoscopy Challenge
We show that depth prediction from synthetic colonoscopy images is robustly solvable, while pose estimation remains an open research question.
M-VAAL: Multimodal Variational Adversarial Active Learning for Downstream Medical Image Analysis Tasks
Acquiring properly annotated data is expensive in the medical field as it requires experts, time-consuming protocols, and rigorous validation.
A spatio-temporal network for video semantic segmentation in surgical videos
The proposed model includes a spatio-temporal decoder to enable video semantic segmentation by improving temporal consistency across frames.
Self-Knowledge Distillation for Surgical Phase Recognition
Conclusion: We embed a self-knowledge distillation framework for the first time in the surgical phase recognition training pipeline.
A Client-server Deep Federated Learning for Cross-domain Surgical Image Segmentation
This paper presents a solution to the cross-domain adaptation problem for 2D surgical image segmentation, explicitly considering the privacy protection of distributed datasets belonging to different centers.
Online estimation of the hand-eye transformation from surgical scenes
To solve the hand-eye problem in robotic-assisted minimally invasive surgery and also simplify the calibration procedure by using neural network method cooporating with the new objective function.
T2FNorm: Extremely Simple Scaled Train-time Feature Normalization for OOD Detection
Indeed, the naive incorporation of feature normalization within neural networks does not guarantee substantial improvement in OOD detection performance.
Surgical tool classification and localization: results and methods from the MICCAI 2022 SurgToolLoc challenge
Unfortunately, obtaining the annotations needed to train machine learning models to identify and localize surgical tools is a difficult task.
Shifted-Windows Transformers for the Detection of Cerebral Aneurysms in Microsurgery
Automated recognition of instances when the aneurysm is exposed in the surgical video would be a valuable reference point for neuronavigation, indicating phase transitioning and more importantly designating moments of high risk for rupture.
3D Generative Model Latent Disentanglement via Local Eigenprojection
Designing realistic digital humans is extremely complex.
Ultrasound Plane Pose Regression: Assessing Generalized Pose Coordinates in the Fetal Brain
In obstetric ultrasound (US) scanning, the learner's ability to mentally build a three-dimensional (3D) map of the fetus from a two-dimensional (2D) US image represents a significant challenge in skill acquisition.
MoBYv2AL: Self-supervised Active Learning for Image Classification
We present MoBYv2AL, a novel self-supervised active learning framework for image classification.
ASPnet: Action Segmentation With Shared-Private Representation of Multiple Data Sources
Most state-of-the-art methods for action segmentation are based on single input modalities or naive fusion of multiple data sources.
Ranked #1 on
Action Segmentation
on 50 Salads
(using extra training data)
Biomedical image analysis competitions: The state of current participation practice
Of these, 84% were based on standard architectures.
Objective Surgical Skills Assessment and Tool Localization: Results from the MICCAI 2021 SimSurgSkill Challenge
Using this publicly available dataset and results as a springboard, future work may enable more efficient training of surgeons with advances in surgical data science.
Generalized Product-of-Experts for Learning Multimodal Representations in Noisy Environments
In order to process the multimodal information automatically and use it for an end application, Multimodal Representation Learning (MRL) has emerged as an active area of research in recent times.
Retrieval of surgical phase transitions using reinforcement learning
In minimally invasive surgery, surgical workflow segmentation from video analysis is a well studied topic.
Learning-Based Keypoint Registration for Fetoscopic Mosaicking
This surgery is minimally invasive and relies on fetoscopy.
BiometryNet: Landmark-based Fetal Biometry Estimation from Standard Ultrasound Planes
Comparison and cross-validation of three different biometric measurements on two independent datasets shows that BiometryNet is robust and yields accurate measurements whose errors are lower than the clinically permissible errors, outperforming other existing automated biometry estimation methods.
Placental Vessel Segmentation and Registration in Fetoscopy: Literature Review and MICCAI FetReg2021 Challenge Findings
For this challenge, we released a dataset of 2060 images, pixel-annotated for vessels, tool, fetus and background classes, from 18 in-vivo TTTS fetoscopy procedures and 18 short video clips.
Fluorescence angiography classification in colorectal surgery -- A preliminary report
A decision boundary was identified based on the tissue classification.
Bimodal Camera Pose Prediction for Endoscopy
Deducing the 3D structure of endoscopic scenes from images is exceedingly challenging.
CholecTriplet2021: A benchmark challenge for surgical action triplet recognition
In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge.
Ranked #1 on
Action Triplet Recognition
on CholecT50 (Challenge)
(using extra training data)
Task-Aware Active Learning for Endoscopic Image Analysis
Semantic segmentation of polyps and depth estimation are two important research problems in endoscopic image analysis.
Histogram of Oriented Gradients Meet Deep Learning: A Novel Multi-task Deep Network for Medical Image Semantic Segmentation
We employed our method on two powerful and widely used semantic segmentation networks: UNet and U2Net to train in a multi-task setup.
Ranked #5 on
Face Identification
on DroneSURF
A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its effect on Image Correspondence
In this paper, we aim at removing specular highlights from endoscopic videos using machine learning.
DeSmoke-LAP: improved unpaired image-to-image translation for desmoking in laparoscopic surgery
Robotic-assisted laparoscopic surgery has become the trend in medicine thanks to its convenience and lower risk of infection against traditional open surgery.
Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation
Automatic surgical scene segmentation is fundamental for facilitating cognitive intelligence in the modern operating theatre.
3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces
Learning a disentangled, interpretable, and structured latent representation in 3D generative models of faces and bodies is still an open problem.
2020 CATARACTS Semantic Segmentation Challenge
In 2020, we released pixel-wise semantic annotations for anatomy and instruments for 4670 images sampled from 25 videos of the CATARACTS training set.
AutoFB: Automating Fetal Biometry Estimation from Standard Ultrasound Planes
To achieve a reproducible and accurate measurement, a sonographer needs to identify three standard 2D planes of the fetal anatomy (head, abdomen, femur) and manually mark the key anatomical landmarks on the image for accurate biometry and fetal weight estimation.
FetReg: Placental Vessel Segmentation and Registration in Fetoscopy Challenge Dataset
Through the \textit{Fetoscopic Placental Vessel Segmentation and Registration (FetReg)} challenge, we present a large-scale multi-centre dataset for the development of generalized and robust semantic segmentation and video mosaicking algorithms for the fetal environment with a focus on creating drift-free mosaics from long duration fetoscopy videos.
Autonomous object harvesting using synchronized optoelectronic microrobots
Optoelectronic tweezer-driven microrobots (OETdMs) are a versatile micromanipulation technology based on the use of light induced dielectrophoresis to move small dielectric structures (microrobots) across a photoconductive substrate.
Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge
Surgical data science is revolutionizing minimally invasive surgery by enabling context-aware applications.
Intrapapillary Capillary Loop Classification in Magnification Endoscopy: Open Dataset and Baseline Methodology
A future work path of high clinical relevance is the extension of the classification to ESCN types.
Image Compositing for Segmentation of Surgical Tools without Manual Annotations
Background data is captured by collecting videos that do not contain instruments.
Gesture Recognition in Robotic Surgery: a Review
This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery focusing on recent data-driven approaches and outlines the open questions and future research directions.
Towards a Computed-Aided Diagnosis System in Colonoscopy: Automatic Polyp Segmentation Using Convolution Neural Networks
We additionally use Shape from-Shading (SfS) to recover depth and provide a richer representation of the tissue's structure in colonoscopy images.
Stereo Correspondence and Reconstruction of Endoscopic Data Challenge
The stereo correspondence and reconstruction of endoscopic data sub-challenge was organized during the Endovis challenge at MICCAI 2019 in Shenzhen, China.
SERV-CT: A disparity dataset from CT for validation of endoscopic 3D reconstruction
The SERV-CT dataset provides an easy to use stereoscopic validation for surgical applications with smooth reference disparities and depths with coverage over the majority of the endoscopic images.
Surgical Data Science -- from Concepts toward Clinical Translation
We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.
Detecting small polyps using a Dynamic SSD-GAN
Endoscopic examinations are used to inspect the throat, stomach and bowel for polyps which could develop into cancer.
Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy
The Endoscopy Computer Vision Challenge (EndoCV) is a crowd-sourcing initiative to address eminent problems in developing reliable computer aided detection and diagnosis endoscopy systems and suggest a pathway for clinical translation of technologies.
Surgical Video Motion Magnification with Suppression of Instrument Artefacts
Video motion magnification could directly highlight subsurface blood vessels in endoscopic video in order to prevent inadvertent damage and bleeding.
Intraoperative Liver Surface Completion with Graph Convolutional VAE
In this work we propose a method based on geometric deep learning to predict the complete surface of the liver, given a partial point cloud of the organ obtained during the surgical laparoscopic procedure.
Real-Time Segmentation of Non-Rigid Surgical Tools based on Deep Learning and Tracking
The latter, a combination of deep learning with optical flow tracking, yields an average balanced accuracy of 78. 2% across all the validated datasets.
Predicting Visual Overlap of Images Through Interpretable Non-Metric Box Embeddings
Even when this is a known scene, the answer typically requires an expensive search across scale space, with matching and geometric verification of large sets of local features.
Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery
Semantic tool segmentation in surgical videos is important for surgical scene understanding and computer-assisted interventions as well as for the development of robotic automation.
Deep Placental Vessel Segmentation for Fetoscopic Mosaicking
We propose a solution utilising the U-Net architecture for performing placental vessel segmentation in fetoscopic videos.
The challenges of deploying artificial intelligence models in a rapidly evolving pandemic
The COVID-19 pandemic, caused by the severe acute respiratory syndrome coronavirus 2, emerged into a world being rapidly transformed by artificial intelligence (AI) based on big data, computational power and neural networks.
Deep iterative vessel segmentation in OCT angiography
This paper addresses retinal vessel segmentation on optical coherence tomography angiography (OCT-A) images of the human retina.
Multi-Task Recurrent Neural Network for Surgical Gesture Recognition and Progress Prediction
Surgical gesture recognition is important for surgical data science and computer-aided intervention.
2018 Robotic Scene Segmentation Challenge
In 2015 we began a sub-challenge at the EndoVis workshop at MICCAI in Munich using endoscope images of ex-vivo tissue with automatically generated annotations from robot forward kinematics and instrument CAD models.
More unlabelled data or label more data? A study on semi-supervised laparoscopic image segmentation
Improving a semi-supervised image segmentation task has the option of adding more unlabelled images, labelling the unlabelled images or combining both, as neither image acquisition nor expert labelling can be considered trivial in most clinical applications.
Weakly Supervised Recognition of Surgical Gestures
Hence, the potential of weak supervision could be to improve unsupervised learning while avoiding manual annotation of large datasets.
Whole-Sample Mapping of Cancerous and Benign Tissue Properties
Structural and mechanical differences between cancerous and healthy tissue give rise to variations in macroscopic properties such as visual appearance and elastic modulus that show promise as signatures for early cancer detection.
Deep Sequential Mosaicking of Fetoscopic Videos
Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time.
CaDIS: Cataract Dataset for Image Segmentation
Video feedback provides a wealth of information about surgical procedures and is the main sensory cue for surgeons.
2D Semantic Segmentation task 1 (8 classes)
2D Semantic Segmentation task 2 (17 classes)
+6
2017 Robotic Instrument Segmentation Challenge
In mainstream computer vision and machine learning, public datasets such as ImageNet, COCO and KITTI have helped drive enormous improvements by enabling researchers to understand the strengths and limitations of different algorithms via performance comparison.
Nonrigid reconstruction of 3D breast surfaces with a low-cost RGBD camera for surgical planning and aesthetic evaluation
Accounting for 26% of all new cancer cases worldwide, breast cancer remains the most common form of cancer in women.
SurReal: enhancing Surgical simulation Realism using style transfer
Surgical simulation is an increasingly important element of surgical education.
FaceOff: Anonymizing Videos in the Operating Rooms
After testing an existing face detection trained model, a new dataset tailored to the surgical environment, with faces obstructed by surgical masks and caps, was collected for fine-tuning to achieve higher face-detection rates in the OR.
Automated Performance Assessment in Transoesophageal Echocardiography with Convolutional Neural Networks
Transoesophageal echocardiography (TEE) is a valuable diagnostic and monitoring imaging modality.
Higher Order of Motion Magnification for Vessel Localisation in Surgical Video
Locating vessels during surgery is critical for avoiding inadvertent damage, yet vasculature can be difficult to identify.
Why rankings of biomedical image analysis competitions should be interpreted with care
International challenges have become the standard for validation of biomedical image analysis methods.
Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery
The paper presents a comparative validation study of different vision-based methods for instrument segmentation and tracking in the context of robotic as well as conventional laparoscopic surgery.
Interpretable Fully Convolutional Classification of Intrapapillary Capillary Loops for Real-Time Detection of Early Squamous Neoplasia
We present a new approach to visualise attention that aims to give some insights on those areas of the oesophageal tissue that lead a network to conclude that the images belong to a particular class and compare them with those visual features employed by clinicians to produce a clinical diagnosis.
Augmented Reality needle ablation guidance tool for Irreversible Electroporation in the pancreas
Irreversible electroporation (IRE) is a soft tissue ablation technique suitable for treatment of inoperable tumours in the pancreas.
Widening siamese architectures for stereo matching
Computational stereo is one of the classical problems in computer vision.
Simultaneous Recognition and Pose Estimation of Instruments in Minimally Invasive Surgery
An additional advantage of our approach is that instrument detection at test time is achieved while avoiding the need for scale-dependent sliding window evaluation.
Refractive Structure-From-Motion Through a Flat Refractive Interface
By explicitly considering a refractive interface, we develop a succinct derivation of the refractive fundamental matrix in the form of the generalised epipolar constraint for an axial camera.
Patch-based adaptive weighting with segmentation and scale (PAWSS) for visual tracking
Tracking-by-detection algorithms are widely used for visual tracking, where the problem is treated as a classification task where an object model is updated over time using online learning techniques.
ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools
We propose the use of parametric rectified linear units for semantic labeling in these small architectures to increase the regularization ability of the design and maintain the segmentation accuracy without overfitting the training sets.
Fast Estimation of Haemoglobin Concentration in Tissue Via Wavelet Decomposition
Tissue oxygenation and perfusion can be an indicator for organ viability during minimally invasive surgery, for example allowing real-time assessment of tissue perfusion and oxygen saturation.
A comparative study of breast surface reconstruction for aesthetic outcome assessment
Breast cancer is the most prevalent cancer type in women, and while its survival rate is generally high the aesthetic outcome is an increasingly important factor when evaluating different treatment alternatives.
Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging
Intra-operative measurements of tissue shape and multi/ hyperspectral information have the potential to provide surgical guidance and decision making support.
Similarity Registration Problems for 2D/3D Ultrasound Calibration
The needle is tracked as a 3D line, and is scanned by the ultrasound as either a 3D line (3D US) or as a 2D point (2D US).
Inference of Haemoglobin Concentration From Stereo RGB
We demonstrate by using the co-registered stereo image data from two cameras it is possible to get robust SO2 estimation as well.
Probe-based Rapid Hybrid Hyperspectral and Tissue Surface Imaging Aided by Fully Convolutional Networks
In HSI mode standard endoscopic illumination is used, with the fibre probe collecting reflected light and encoding the spatial information into a linear format that can be imaged onto the slit of a spectrograph.
