Search Results for author:
Found 37 papers, 18 papers with code
fine-CLIP: Enhancing Zero-Shot Fine-Grained Surgical Action Recognition with Vision-Language Models
While vision-language models like CLIP have advanced zero-shot surgical phase recognition, they struggle with fine-grained surgical activities, especially action triplets.
Learning from Synchronization: Self-Supervised Uncalibrated Multi-View Person Association in Challenging Scenes
Specifically, we propose a self-supervised learning framework, consisting of an encoder-decoder model and a self-supervised pretext task, cross-view image synchronization, which aims to distinguish whether two images from different views are captured at the same time.
S4M: Segment Anything with 4 Extreme Points
To address this, we introduce S4M (Segment Anything with 4 Extreme Points), which augments SAM by leveraging extreme points -- the top-, bottom-, left-, and right-most points of an instance -- prompts.
Multi-view Video-Pose Pretraining for Operating Room Surgical Activity Recognition
We highlight the benefits of our approach for surgical activity recognition in both multi-view and single-view settings, showcasing its practical applicability in complex surgical environments.
When do they StOP?: A First Step Towards Automatically Identifying Team Communication in the Operating Room
Conclusion: We investigate the Team Time-Out and the StOP?-protocol in the OR, by presenting the first OR dataset with temporal annotations of group activities protocols, and introducing a novel group activity detection approach that outperforms existing approaches.
Early Operative Difficulty Assessment in Laparoscopic Cholecystectomy via Snapshot-Centric Video Analysis
Our results on the CholeScore dataset establishes a new benchmark to study LCOD using intraoperative video data.
UltraSam: A Foundation Model for Ultrasound using Large Open-Access Segmentation Datasets
Methods: We compile US-43d, a large-scale collection of 43 open-access ultrasound datasets with over 280, 000 images and segmentation masks for more than 50 anatomical structures.
CycleSAM: One-Shot Surgical Scene Segmentation using Cycle-Consistent Feature Matching to Prompt SAM
The recently introduced Segment-Anything Model (SAM) has the potential to greatly accelerate the development of segmentation models.
Overcoming Dimensional Collapse in Self-supervised Contrastive Learning for Medical Image Segmentation
In this work, we investigate the application of contrastive learning to the domain of medical image analysis.
ST(OR)2: Spatio-Temporal Object Level Reasoning for Activity Recognition in the Operating Room
Surgical robotics holds much promise for improving patient safety and clinician experience in the Operating Room (OR).
The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmark
This technical report provides a detailed overview of Endoscapes, a dataset of laparoscopic cholecystectomy (LC) videos with highly intricate annotations targeted at automated assessment of the Critical View of Safety (CVS).
Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgery
The use of multi-centric training data, experiments 6) and 7), improves the generalization capabilities of the models, bringing them beyond the level of independent mono-centric training and validation (experiments 1) and 2)).
Encoding Surgical Videos as Latent Spatiotemporal Graphs for Object and Anatomy-Driven Reasoning
Recently, spatiotemporal graphs have emerged as a concise and elegant manner of representing video clips in an object-centric fashion, and have shown to be useful for downstream tasks such as action recognition.
TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research
To validate the dataset's utility, 5 competitive Deep Learning models for automatic kidney segmentation were benchmarked, yielding average DICE scores from 83. 2% to 89. 1% for CT, and 61. 9% to 79. 4% for US images.
Surgical Action Triplet Detection by Mixed Supervised Learning of Instrument-Tissue Interactions
We analyze how the amount of instrument spatial annotations affects triplet detection and observe that accurate instrument localization does not guarantee better triplet detection due to the risk of erroneous associations with the verbs and targets.
Weakly Supervised Temporal Convolutional Networks for Fine-grained Surgical Activity Recognition
In this work, we propose to use coarser and easier-to-annotate activity labels, namely phases, as weak supervision to learn step recognition with fewer step annotated videos.
CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection.
Ranked #1 on
Action Triplet Detection
on CholecT50 (Challenge)
Preserving Privacy in Surgical Video Analysis Using Artificial Intelligence: A Deep Learning Classifier to Identify Out-of-Body Scenes in Endoscopic Videos
Results: The internal dataset consisting of 356, 267 images from 48 videos and the two multicentric test datasets consisting of 54, 385 and 58, 349 images from 10 and 20 videos, respectively, were annotated.
Real-Time Artificial Intelligence Assistance for Safe Laparoscopic Cholecystectomy: Early-Stage Clinical Evaluation
Artificial intelligence is set to be deployed in operating rooms to improve surgical care.
Latent Graph Representations for Critical View of Safety Assessment
Assessing the critical view of safety in laparoscopic cholecystectomy requires accurate identification and localization of key anatomical structures, reasoning about their geometric relationships to one another, and determining the quality of their exposure.
Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognition
Focusing more on the verbs, our RiT explores the connectedness of current and past frames to learn temporal attention-based features for enhanced triplet recognition.
Ranked #2 on
Action Triplet Recognition
on CholecT45 (cross-val)
CholecTriplet2021: A benchmark challenge for surgical action triplet recognition
In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge.
Ranked #1 on
Action Triplet Recognition
on CholecT50 (Challenge)
(using extra training data)
Live Laparoscopic Video Retrieval with Compressed Uncertainty
Searching through large volumes of medical data to retrieve relevant information is a challenging yet crucial task for clinical care.
Temporally Constrained Neural Networks (TCNN): A framework for semi-supervised video semantic segmentation
A major obstacle to building models for effective semantic segmentation, and particularly video semantic segmentation, is a lack of large and well annotated datasets.
Ranked #2 on
Semantic Segmentation
on Endoscapes
Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos
To achieve this task, we introduce our new model, the Rendezvous (RDV), which recognizes triplets directly from surgical videos by leveraging attention at two different levels.
Ranked #1 on
Action Triplet Recognition
on CholecT50
Intraoperative time out to promote the implementation of the critical view of safety in laparoscopic cholecystectomy: a video-based assessment of 343 procedures
The primary endpoint was to compare the rate of CVS achievement between LCs performed in the year before and the year after the 5-second rule.
Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass Procedures
Conclusion: In this work, we present a multi-task multi-stage temporal convolutional network for surgical activity recognition, which shows improved results compared to single-task models on the Bypass40 gastric bypass dataset with multi-level annotations.
Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets
Recognition of surgical activity is an essential component to develop context-aware decision support for the operating room.
Ranked #1 on
Action Triplet Recognition
on CholecT40
Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos
Results: We build a baseline tracker on top of the CNN model and demonstrate that our approach based on the ConvLSTM outperforms the baseline in tool presence detection, spatial localization, and motion tracking by over 5. 0%, 13. 9%, and 12. 6%, respectively.
Ranked #2 on
Surgical tool detection
on Cholec80
Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition
Vision algorithms capable of interpreting scenes from a real-time video stream are necessary for computer-assisted surgery systems to achieve context-aware behavior.
Future-State Predicting LSTM for Early Surgery Type Recognition
This work presents a novel approach for the early recognition of the type of a laparoscopic surgery from its video.
Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos
We propose a deep architecture, trained solely on image level annotations, that can be used for both tool presence detection and localization in surgical videos.
Ranked #4 on
Surgical tool detection
on Cholec80
Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networks
In this work, we propose a new self-supervised pre-training approach based on the prediction of remaining surgery duration (RSD) from laparoscopic videos.
RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations
In this paper, we propose a deep learning pipeline, referred to as RSDNet, which automatically estimates the remaining surgery duration (RSD) intraoperatively by using only visual information from laparoscopic videos.
Single- and Multi-Task Architectures for Tool Presence Detection Challenge at M2CAI 2016
The tool presence detection challenge at M2CAI 2016 consists of identifying the presence/absence of seven surgical tools in the images of cholecystectomy videos.
Single- and Multi-Task Architectures for Surgical Workflow Challenge at M2CAI 2016
On top of these architectures we propose to use two different approaches to enforce the temporal constraints of the surgical workflow: (1) HMM-based and (2) LSTM-based pipelines.
EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos
In the literature, two types of features are typically used to perform this task: visual features and tool usage signals.
Ranked #5 on
Surgical tool detection
on Cholec80
Offline surgical phase recognition
Online surgical phase recognition
+2
