Search Results for author:
Found 62 papers, 24 papers with code
SAGE: SLAM with Appearance and Geometry Prior for Endoscopy
In endoscopy, many applications (e. g., surgical navigation) would benefit from a real-time method that can simultaneously track the endoscope and reconstruct the dense 3D geometry of the observed anatomy from a monocular endoscopic video.
"Good Robot! Now Watch This!": Repurposing Reinforcement Learning for Task-to-Task Transfer
Based upon this insight, we propose See-SPOT-Run (SSR), a new computational approach to robot learning that enables a robot to complete a variety of real robot tasks in novel problem domains without task-specific training.
Learn Proportional Derivative Controllable Latent Space from Pixels
Recent advances in latent space dynamics model from pixels show promising progress in vision-based model predictive control (MPC).
Neighborhood Normalization for Robust Geometric Feature Learning
Extracting geometric features from 3D models is a common first step in applications such as 3D registration, tracking, and scene flow estimation.
Localization and Control of Magnetic Suture Needles in Cluttered Surgical Site with Blood and Tissue
Our localization method combines neural network-based segmentation and classical techniques, and we are able to consistently locate our needle with 0. 73 mm RMS error in clean environments and 2. 72 mm RMS error in challenging environments with blood and occlusion.
Single View Geocentric Pose in the Wild
Current methods for Earth observation tasks such as semantic mapping, map alignment, and change detection rely on near-nadir images; however, often the first available images in response to dynamic world events such as natural disasters are oblique.
Motion Guided Attention Fusion to Recognize Interactions from Videos
We present a dual-pathway approach for recognizing fine-grained interactions from videos.
"Train one, Classify one, Teach one" -- Cross-surgery transfer learning for surgical step recognition
Such pre-training enables TSAN to learn workflow steps of a new laparoscopic procedure type from only a small number of labeled samples from the target procedure.
SAFCAR: Structured Attention Fusion for Compositional Action Recognition
We present a general framework for compositional action recognition -- i. e. action recognition where the labels are composed out of simpler components such as subjects, atomic-actions and objects.
Fine-grained activity recognition for assembly videos
In this paper we address the task of recognizing assembly actions as a structure (e. g. a piece of furniture or a toy block tower) is built up from a set of primitive objects.
Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation
We propose an unsupervised vision-based system to estimate the joint configurations of the robot arm from a sequence of RGB or RGB-D images without knowing the model a priori, and then adapt it to the task of category-independent articulated object pose estimation.
Autonomously Navigating a Surgical Tool Inside the Eye by Learning from Demonstration
We show that the network can reliably navigate a needle surgical tool to various desired locations within 137 microns accuracy in physical experiments and 94 microns in simulation on average, and generalizes well to unseen situations such as in the presence of auxiliary surgical tools, variable eye backgrounds, and brightness conditions.
Surgical Data Science -- from Concepts toward Clinical Translation
We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.
Deep Hiearchical Multi-Label Classification Applied to Chest X-Ray Abnormality Taxonomies
To this end, we present a deep HMLC approach for CXR CAD.
Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision
We use three different methods to manipulate these latent representations in order to predict tool presence in each frame.
Anatomy-Aware Siamese Network: Exploiting Semantic Asymmetry for Accurate Pelvic Fracture Detection in X-ray Images
A new contrastive feature learning component in our Siamese network is designed to optimize the deep image features being more salient corresponding to the underlying semantic asymmetries (caused by pelvic fracture occurrences).
Learning Geocentric Object Pose in Oblique Monocular Images
An object's geocentric pose, defined as the height above ground and orientation with respect to gravity, is a powerful representation of real-world structure for object detection, segmentation, and localization tasks using RGBD images.
Artificial Intelligence-based Clinical Decision Support for COVID-19 -- Where Art Thou?
The COVID-19 crisis has brought about new clinical questions, new workflows, and accelerated distributed healthcare needs.
Semantic Image Manipulation Using Scene Graphs
In our work, we address the novel problem of image manipulation from scene graphs, in which a user can edit images by merely applying changes in the nodes or edges of a semantic graph that is generated from the image.
Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment
Reconstructing accurate 3D surface models of sinus anatomy directly from an endoscopic video is a promising avenue for cross-sectional and longitudinal analysis to better understand the relationship between sinus anatomy and surgical outcomes.
Extremely Dense Point Correspondences using a Learned Feature Descriptor
In direct comparison to recent local and dense descriptors on an in-house sinus endoscopy dataset, we demonstrate that our proposed dense descriptor can generalize to unseen patients and scopes, thereby largely improving the performance of Structure from Motion (SfM) in terms of model density and completeness.
Accurate Detection of Out of Body Segments in Surgical Video using Semi-Supervised Learning
Training a deep model to detect out-of-body and non-relevant segments in surgical videos requires suitable labeling.
Car Pose in Context: Accurate Pose Estimation with Ground Plane Constraints
Scene context is a powerful constraint on the geometry of objects within the scene in cases, such as surveillance, where the camera geometry is unknown and image quality may be poor.
DASZL: Dynamic Action Signatures for Zero-shot Learning
There are many realistic applications of activity recognition where the set of potential activity descriptions is combinatorially large.
RSA: Randomized Simulation as Augmentation for Robust Human Action Recognition
In this paper, we propose the Randomized Simulation as Augmentation (RSA) framework which augments real-world training data with synthetic data to improve the robustness of action recognition networks.
Action Recognition Using Volumetric Motion Representations
In this work, we introduce a novel representation of motion as a voxelized 3D vector field and demonstrate how it can be used to improve performance of action recognition networks.
"Good Robot!": Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer
We are able to create real stacks in 100% of trials with 61% efficiency and real rows in 100% of trials with 59% efficiency by directly loading the simulation-trained model on the real robot with no additional real-world fine-tuning.
Self-supervised Dense 3D Reconstruction from Monocular Endoscopic Video
We present a self-supervised learning-based pipeline for dense 3D reconstruction from full-length monocular endoscopic videos without a priori modeling of anatomy or shading.
Automated Surgical Activity Recognition with One Labeled Sequence
Prior work has demonstrated the feasibility of automated activity recognition in robot-assisted surgery from motion data.
sharpDARTS: Faster and More Accurate Differentiable Architecture Search
We have performed an in-depth analysis to identify limitations in a widely used search space and a recent architecture search method, Differentiable Architecture Search (DARTS).
Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods
We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading.
Semantic Stereo for Incidental Satellite Images
The increasingly common use of incidental satellite images for stereo reconstruction versus rigidly tasked binocular or trinocular coincident collection is helping to enable timely global-scale 3D mapping; however, reliable stereo correspondence from multi-date image pairs remains very challenging due to seasonal appearance differences and scene change.
The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints
We show that a mild relaxation of the task and workspace constraints implicit in existing object grasping datasets can cause neural network based grasping algorithms to fail on even a simple block stacking task when executed under more realistic circumstances.
Towards automatic initialization of registration algorithms using simulated endoscopy images
Several registration algorithms have been developed, many of which achieve high accuracy.
Endoscopic navigation in the absence of CT imaging
Clinical examinations that involve endoscopic exploration of the nasal cavity and sinuses often do not have a reference image to provide structural context to the clinician.
Unsupervised Learning for Surgical Motion by Learning to Predict the Future
We show that it is possible to learn meaningful representations of surgical motion, without supervision, by learning to predict the future.
Guide Me: Interacting with Deep Networks
Interaction and collaboration between humans and intelligent machines has become increasingly important as machine learning methods move into real-world applications that involve end users.
Visual Robot Task Planning
In this work, we propose a neural network architecture and associated planning algorithm that (1) learns a representation of the world useful for generating prospective futures after the application of high-level actions, (2) uses this generative model to simulate the result of sequences of high-level actions in a variety of environments, and (3) uses this same representation to evaluate these actions and perform tree search to find a sequence of high-level actions in a new environment.
A Unified Framework for Multi-View Multi-Class Object Pose Estimation
To learn discriminative pose features, we integrate three new capabilities into a deep Convolutional Neural Network (CNN): an inference scheme that combines both classification and pose regression based on a uniform tessellation of the Special Euclidean group in three dimensions (SE(3)), the fusion of class priors into the training process via a tiled class map, and an additional regularization using deep supervision with an object mask.
Occupancy Map Prediction Using Generative and Fully Convolutional Networks for Vehicle Navigation
In these situations, the robot's ability to reason about its future motion is often severely limited by sensor field of view (FOV).
Deep Supervision with Intermediate Concepts
In this work, we explore an approach for injecting prior domain structure into neural network training by supervising hidden layers of a CNN with intermediate concepts that normally are not observed in practice.
Predicting Multiple Actions for Stochastic Continuous Control
We introduce a new approach to estimate continuous actions using actor-critic algorithms for reinforcement learning problems.
Learning to Imagine Manipulation Goals for Robot Task Planning
Ideally, we would combine the ability of machine learning to leverage big data for learning the semantics of a task, while using techniques from task planning to reliably generalize to new environment.
Adversarial Deep Structured Nets for Mass Segmentation from Mammograms
Mass segmentation provides effective morphological features which are important for mass diagnosis.
Temporal and Physical Reasoning for Perception-Based Robotic Manipulation
Accurate knowledge of object poses is crucial to successful robotic manipulation tasks, and yet most current approaches only work in laboratory settings.
Robotics
Advances in Artificial Intelligence Require Progress Across all of Computer Science
Advances in Artificial Intelligence require progress across all of computer science.
Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments
This paper investigates the ability of neural networks to learn both LTL constraints and control policies in order to generate task plans in complex environments.
Robotics
Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies
Recurrent neural networks (RNNs) have achieved state-of-the-art performance on many diverse tasks, from machine translation to surgical activity recognition, yet training RNNs to capture long-term dependencies remains difficult.
Regularizing Face Verification Nets For Pain Intensity Regression
In this way, the expression intensity regression task can benefit from the rich feature representations trained on a huge amount of data for face verification.
Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing
Monocular 3D object parsing is highly desirable in various scenarios including occlusion reasoning and holistic scene interpretation.
Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
In future prediction, for example, many distinct outcomes are equally valid.
Temporal Convolutional Networks for Action Segmentation and Detection
The ability to identify and temporally segment fine-grained human actions throughout a video is crucial for robotics, surveillance, education, and beyond.
Anatomically Constrained Video-CT Registration via the V-IMLOP Algorithm
Functional endoscopic sinus surgery (FESS) is a surgical procedure used to treat acute cases of sinusitis and other sinus diseases.
Temporal Convolutional Networks: A Unified Approach to Action Segmentation
The dominant paradigm for video-based action segmentation is composed of two steps: first, for each frame, compute low-level features using Dense Trajectories or a Convolutional Neural Network that encode spatiotemporal information locally, and second, input these features into a classifier that captures high-level temporal relationships, such as a Recurrent Neural Network (RNN).
Ranked #6 on
Action Segmentation
on JIGSAWS
SANTIAGO: Spine Association for Neuron Topology Improvement and Graph Optimization
Our automated approach improves the local subgraph score by more than four times and the full graph score by 60 percent.
Recognizing Surgical Activities with Recurrent Neural Networks
In contrast, we work on recognizing both gestures and longer, higher-level activites, or maneuvers, and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks.
Ranked #1 on
Surgical Skills Evaluation
on MISTIC-SIL
Segmental Spatiotemporal CNNs for Fine-grained Action Segmentation
We propose a model for action segmentation which combines low-level spatiotemporal features with a high-level segmental classifier.
Ranked #7 on
Action Segmentation
on JIGSAWS
Beyond Spatial Pooling: Fine-Grained Representation Learning in Multiple Domains
In this paper, we formulate a probabilistic framework for analyzing the performance of pooling.
Automated Objective Surgical Skill Assessment in the Operating Room Using Unstructured Tool Motion
Previous work on surgical skill assessment using intraoperative tool motion in the operating room (OR) has focused on highly-structured surgical tasks such as cholecystectomy.
An Automated Images-to-Graphs Framework for High Resolution Connectomics
Finally, we deploy a reference end-to-end version of the pipeline on a large, publicly available data set.
Hierarchical Sparse and Collaborative Low-Rank Representation for Emotion Recognition
In this paper, we design a Collaborative-Hierarchical Sparse and Low-Rank (C-HiSLR) model that is natural for recognizing human emotion in visual data.
VESICLE: Volumetric Evaluation of Synaptic Interfaces using Computer vision at Large Scale
An open challenge problem at the forefront of modern neuroscience is to obtain a comprehensive mapping of the neural pathways that underlie human brain function; an enhanced understanding of the wiring diagram of the brain promises to lead to new breakthroughs in diagnosing and treating neurological disorders.
