Search Results for author:
Found 123 papers, 70 papers with code
O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning
Building upon our previous investigations of O1 replication (Part 1: Journey Learning [Qin et al., 2024] and Part 2: Distillation [Huang et al., 2024]), this work explores the potential of inference-time scaling in large language models (LLMs) for medical reasoning tasks, ranging from diagnostic decision-making to treatment planning.
GLFC: Unified Global-Local Feature and Contrast Learning with Mamba-Enhanced UNet for Synthetic CT Generation from CBCT
In this work, we propose a Global-Local Feature and Contrast learning (GLFC) framework for sCT generation.
Guiding Medical Vision-Language Models with Explicit Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations
To the best of our knowledge, this is the first work to explicitly introduce visual prompt into medical VLMs, and we successfully outperform recent state-of-the-art large models across multiple medical VQA datasets.
Cycle-Consistent Bridge Diffusion Model for Accelerated MRI Reconstruction
CBDM employs two bridge diffusion models to construct a cycle-consistent diffusion process with a consistency loss, enhancing the fine-grained details of reconstructed images and reducing the number of diffusion steps.
Cross Group Attention and Group-wise Rolling for Multimodal Medical Image Synthesis
Multimodal MR image synthesis aims to generate missing modality image by fusing and mapping a few available MRI data.
Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment
Recent advancements in medical vision-language pre-training models have driven significant progress in zero-shot disease recognition.
MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications
This paper introduces a diffusion-based foundation model to address a diverse range of medical image tasks, namely MedDiff-FM.
MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling
MeNTi integrates a specialized medical toolkit and employs meta-tool and nested calling mechanisms to enhance LLM tool utilization.
BrainMVP: Multi-modal Vision Pre-training for Brain Image Analysis using Multi-parametric MRI
Then, a novel pre-training paradigm is proposed for the multi-modal MRI data, addressing the issue of missing modalities and achieving multi-modal information fusion.
A foundation model for generalizable disease diagnosis in chest X-ray images
Medical artificial intelligence (AI) is revolutionizing the interpretation of chest X-ray (CXR) images by providing robust tools for disease diagnosis.
MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generation
Medicine is inherently multimodal and multitask, with diverse data modalities spanning text, imaging.
DeReStainer: H&E to IHC Pathological Image Translation via Decoupled Staining Channels
Breast cancer is a highly fatal disease among cancers in women, and early detection is crucial for treatment.
GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI
Large Vision-Language Models (LVLMs) are capable of handling diverse data types such as imaging, text, and physiological signals, and can be applied in various fields.
Multi-modal Data Binding for Survival Analysis Modeling with Incomplete Data and Annotations
Survival analysis stands as a pivotal process in cancer treatment research, crucial for predicting patient survival rates accurately.
Cost-effective Instruction Learning for Pathology Vision and Language Analysis
Here we propose a cost-effective instruction learning framework for conversational pathology named as CLOVER.
CC-DCNet: Dynamic Convolutional Neural Network with Contrastive Constraints for Identifying Lung Cancer Subtypes on Multi-modality Images
To this end, we propose a novel deep learning network designed to accurately classify lung cancer subtype with multi-dimensional and multi-modality images, i. e., CT and pathological images.
Pathology-knowledge Enhanced Multi-instance Prompt Learning for Few-shot Whole Slide Image Classification
Prompt learning based on the pre-trained models (\eg, CLIP) appears to be a promising scheme for this setting; however, current research in this area is limited, and existing algorithms often focus solely on patch-level prompts or confine themselves to language prompts.
MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Models
In this work, we introduce "MedBench", a comprehensive, standardized, and reliable benchmarking system for Chinese medical LLM.
MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokens
Some benchmarks in the generic domain have also followed up on evaluating long-context capabilities.
Rethinking Abdominal Organ Segmentation (RAOS) in the clinical scenario: A robustness evaluation benchmark with challenging cases
We also assessed cross-generalization between RAOS and three public datasets.
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
We delve into the models' cognitive reasoning abilities, their performance across different modalities, and their outcomes in process-level evaluations, which are vital for tasks requiring complex reasoning with lengthy solutions.
A Comprehensive Survey of Foundation Models in Medicine
In this survey, we present a comprehensive review of FMs in medicine, focusing on their evolution, learning strategies, flagship models, applications, and associated challenges.
CAT: Coordinating Anatomical-Textual Prompts for Multi-Organ and Tumor Segmentation
Existing promptable segmentation methods in the medical imaging field primarily consider either textual or visual prompts to segment relevant objects, yet they often fall short when addressing anomalies in medical images, like tumors, which may vary greatly in shape, size, and appearance.
Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges
As a result, an overview of the categories, methods, and evaluation of medical dialogue systems remain limited and underspecified, hindering the further improvement of this area.
Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models
With experimental results, we show that our framework brings notable performance improvements and surpasses the previous counterparts in the evidence retrieval process in terms of evidence retrieval accuracy.
FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation
Adapting a medical image segmentation model to a new domain is important for improving its cross-domain transferability, and due to the expensive annotation process, Unsupervised Domain Adaptation (UDA) is appealing where only unlabeled images are needed for the adaptation.
PathoTune: Adapting Visual Foundation Model to Pathological Specialists
As natural image understanding moves towards the pretrain-finetune era, research in pathology imaging is concurrently evolving.
VLM-CPL: Consensus Pseudo Labels from Vision-Language Models for Human Annotation-Free Pathological Image Classification
To address this issue, we introduce VLM-CPL, a novel approach based on consensus pseudo labels that integrates two noisy label filtering techniques with a semi-supervised learning strategy.
Modality-Aware and Shift Mixer for Multi-modal Brain Tumor Segmentation
Combining images from multi-modalities is beneficial to explore various information in computer vision, especially in the medical domain.
OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine
The emerging trend of advancing generalist artificial intelligence, such as GPTv4 and Gemini, has reshaped the landscape of research (academia and industry) in machine learning and many other research areas.
Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining
However, it is challenging for existing methods to model long-range global information, where convolutional neural networks (CNNs) are constrained by their local receptive fields, and vision transformers (ViTs) suffer from high quadratic complexity of their attention mechanism.
Evaluating and Enhancing Large Language Models Performance in Domain-specific Medicine: Osteoarthritis Management with DocOA
This study focused on evaluating and enhancing the clinical capabilities of LLMs in specific domains, using osteoarthritis (OA) management as a case study.
Data-Centric Foundation Models in Computational Healthcare: A Survey
The advent of foundation models (FMs) as an emerging suite of AI techniques has struck a wave of opportunities in computational healthcare.
SegRap2023: A Benchmark of Organs-at-Risk and Gross Tumor Volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma
The precise delineation of Gross Tumor Volumes (GTVs) and Organs-At-Risk (OARs) is crucial in radiation treatment, directly impacting patient prognosis.
ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Prompting
The long-tailed distribution problem in medical image analysis reflects a high prevalence of common conditions and a low prevalence of rare ones, which poses a significant challenge in developing a unified model capable of identifying rare or novel tumor categories not encountered during training.
Enhancing Visual Grounding and Generalization: A Multi-Task Cycle Training Approach for Vision-Language Models
Furthermore, we contribute a VG dataset, especially with multi-tasks.
SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million masks
Segment Anything Model (SAM) has achieved impressive results for natural image segmentation with input prompts such as points and bounding boxes.
Learning A Multi-Task Transformer Via Unified And Customized Instruction Tuning For Chest Radiograph Interpretation
We first compose a multi-task training dataset comprising 13. 4 million instruction and ground-truth pairs (with approximately one million radiographs) for the customized tuning, involving both image- and pixel-level tasks.
SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images
In this paper, we introduce SAM-Med3D for general-purpose segmentation on volumetric medical images.
Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Prompts
Image synthesis approaches, e. g., generative adversarial networks, have been popular as a form of data augmentation in medical image analysis tasks.
UPL-SFDA: Uncertainty-aware Pseudo Label Guided Source-Free Domain Adaptation for Medical Image Segmentation
The different predictions in these duplicated heads are used to obtain pseudo labels for unlabeled target-domain images and their uncertainty to identify reliable pseudo labels.
Scribble-based 3D Multiple Abdominal Organ Segmentation via Triple-branch Multi-dilated Network with Pixel- and Class-wise Consistency
Specifically, we employ a Triple-branch multi-Dilated network (TDNet) with one encoder and three decoders using different dilation rates to capture features from different receptive fields that are complementary to each other to generate high-quality soft pseudo labels.
Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning
It designs a correlation weighting mechanism to adjust the correlation between masked chest X-ray image patches and their corresponding reports, thereby enhancing the model's representation learning capabilities.
A-Eval: A Benchmark for Cross-Dataset Evaluation of Abdominal Multi-Organ Segmentation
To address these questions, we introduce A-Eval, a benchmark for the cross-dataset Evaluation ('Eval') of Abdominal ('A') multi-organ segmentation.
LLM-Mini-CEX: Automatic Evaluation of Large Language Model for Diagnostic Conversation
Despite their alluring technological potential, there is no unified and comprehensive evaluation criterion, leading to the inability to evaluate the quality and potential risks of medical LLMs, further hindering the application of LLMs in medical treatment scenarios.
Classification of lung cancer subtypes on CT images with synthetic pathological priors
The accurate diagnosis on pathological subtypes for lung cancer is of significant importance for the follow-up treatments and prognosis managements.
Text-guided Foundation Model Adaptation for Pathological Image Classification
In this paper, we propose to Connect Image and Text Embeddings (CITE) to enhance pathological image classification.
Pathology-and-genomics Multimodal Transformer for Survival Outcome Prediction
Survival outcome assessment is challenging and inherently associated with multiple clinical factors (e. g., imaging and genomics biomarkers) in cancer.
Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train
Foundation models have exhibited remarkable success in various applications, such as disease diagnosis and text report generation.
MIS-FM: 3D Medical Image Segmentation using Foundation Models Pretrained on a Large-Scale Unannotated Dataset
The proposed model was pretrained with 110k unannotated 3D CT volumes, and experiments with different downstream segmentation targets including head and neck organs, thoracic/abdominal organs showed that our pretrained model largely outperformed training from scratch and several state-of-the-art self-supervised training methods and segmentation models.
MedLSAM: Localize and Segment Anything Model for 3D CT Images
Our findings are twofold: 1) MedLAM can directly localize anatomical structures using just a few template scans, achieving performance comparable to fully supervised models; 2) MedLSAM closely matches the performance of SAM and its specialized medical adaptations with manual prompts, while minimizing the need for extensive point annotations across the entire dataset.
Exploring Data Redundancy in Real-world Image Classification through Data Selection
Deep learning models often require large amounts of data for training, leading to increased costs.
KiUT: Knowledge-injected U-Transformer for Radiology Report Generation
Radiology report generation aims to automatically generate a clinically accurate and coherent paragraph from the X-ray image, which could relieve radiologists from the heavy burden of report writing.
UM-CAM: Uncertainty-weighted Multi-resolution Class Activation Maps for Weakly-supervised Fetal Brain Segmentation
To this end, we propose a novel weakly-supervised method with image-level labels based on semantic features and context information exploration.
MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification
Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications.
On the Challenges and Perspectives of Foundation Models for Medical Image Analysis
This article discusses the opportunities, applications and future directions of large-scale pre-trained models, i. e., foundation models, for analyzing medical images.
MidMed: Towards Mixed-Type Dialogues for Medical Consultation
To mitigate this challenge, we propose a novel task and create a human-to-human mixed-type medical consultation dialogue corpus, termed MidMed, covering five dialogue types: task-oriented dialogue for diagnosis, recommendation, knowledge-grounded dialogue, QA, and chitchat.
Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation
Inspired by the training program of medical radiology residents, we propose a shift towards universal medical image segmentation, a paradigm aiming to build medical image understanding foundation models by leveraging the diversity and commonality across clinical targets, body regions, and imaging modalities.
Semi-supervised Pathological Image Segmentation via Cross Distillation of Multiple Attentions
In this paper, we propose a novel SSL method based on Cross Distillation of Multiple Attentions (CDMA) to effectively leverage unlabeled images.
ConES: Concept Embedding Search for Parameter Efficient Tuning Large Vision Language Models
To achieve this, we propose a Concept Embedding Search (ConES) approach by optimizing prompt embeddings -- without the need of the text encoder -- to capture the 'concept' of the image modality through a variety of task objectives.
Voxel2Hemodynamics: An End-to-end Deep Learning Method for Predicting Coronary Artery Hemodynamics
In this work, we proposed an end-to-end deep learning framework, which could predict the coronary artery hemodynamics from CCTA images.
MedGPTEval: A Dataset and Benchmark to Evaluate Responses of Large Language Models in Medicine
The responses generated by chatbots based on LLMs are recorded for blind evaluations by five licensed medical experts.
Segmentation and Vascular Vectorization for Coronary Artery by Geometry-based Cascaded Neural Network
3) A dataset named CCA-200 is collected, consisting of 200 CCTA images with coronary artery disease.
Alleviating Exposure Bias via Multi-level Contrastive Learning and Deviation Simulation in Abstractive Summarization
Most Transformer based abstractive summarization systems have a severe mismatch between training and inference, i. e., exposure bias.
SAM on Medical Images: A Comprehensive Study on Three Prompt Modes
As the first promptable foundation model for segmentation tasks, it was trained on a large dataset with an unprecedented number of images and annotations.
STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training
However, the state-of-the-art models for medical image segmentation are still small-scale, with their parameters only in the tens of millions.
Towards General Purpose Medical AI: Continual Learning Medical Foundation Model
Therefore, in this work, we further explore the possibility of leveraging pre-trained VLMs as medical foundation models for building general-purpose medical AI, where we thoroughly investigate three machine-learning paradigms, i. e., domain/task-specialized learning, joint learning, and continual learning, for training the VLMs and evaluate their generalization performance on cross-domain and cross-task test sets.
CDDSA: Contrastive Domain Disentanglement and Style Augmentation for Generalizable Medical Image Segmentation
First, a disentangle network is proposed to decompose an image into a domain-invariant anatomical representation and a domain-specific style code, where the former is sent to a segmentation model that is not affected by the domain shift, and the disentangle network is regularized by a decoder that combines the anatomical and style codes to reconstruct the input image.
PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation
Existing toolkits mainly focus on fully supervised segmentation and require full and accurate pixel-level annotations that are time-consuming and difficult to acquire for segmentation tasks, which makes learning from imperfect labels highly desired for reducing the annotation cost.
Contrastive Semi-supervised Learning for Domain Adaptive Segmentation Across Similar Anatomical Structures
To solve this problem, we propose Contrastive Semi-supervised learning for Cross Anatomy Domain Adaptation (CS-CADA) that adapts a model to segment similar structures in a target domain, which requires only limited annotations in the target domain by leveraging a set of existing annotated images of similar structures in a source domain.
PA-Seg: Learning from Point Annotations for 3D Medical Image Segmentation using Contextual Regularization and Cross Knowledge Distillation
The success of Convolutional Neural Networks (CNNs) in 3D medical image segmentation relies on massive fully annotated 3D volumes for training that are time-consuming and labor-intensive to acquire.
Towards Self-supervised and Weight-preserving Neural Architecture Search
In this work, we propose the self-supervised and weight-preserving neural architecture search (SSWP-NAS) as an extension of the current NAS framework by allowing the self-supervision and retaining the concomitant weights discovered during the search stage.
HMRNet: High and Multi-Resolution Network with Bidirectional Feature Calibration for Brain Structure Segmentation in Radiotherapy
Accurate segmentation of Anatomical brain Barriers to Cancer spread (ABCs) plays an important role for automatic delineation of Clinical Target Volume (CTV) of brain tumors in radiotherapy.
Contrastive Domain Disentanglement for Generalizable Medical Image Segmentation
To tackle this deficiency, we propose Contrastive Domain Disentangle (CDD) network for generalizable medical image segmentation.
Scribble-Supervised Medical Image Segmentation via Dual-Branch Network and Dynamically Mixed Pseudo Labels Supervision
Medical image segmentation plays an irreplaceable role in computer-assisted diagnosis, treatment planning, and following-up.
A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark
Transformers have demonstrated remarkable performance in natural language processing and computer vision.
Graph Convolutional Networks for Multi-modality Medical Imaging: Methods, Architectures, and Clinical Applications
Image-based characterization and disease understanding involve integrative analysis of morphological, spatial, and topological information across biological scales.
Contrastive and Selective Hidden Embeddings for Medical Image Segmentation
Medical image segmentation has been widely recognized as a pivot procedure for clinical diagnosis, analysis, and treatment planning.
Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer
Notably, this work may be the first attempt to combine CNN and transformer for semi-supervised medical image segmentation and achieve promising results on a public benchmark.
One-shot Weakly-Supervised Segmentation in Medical Images
Deep neural networks usually require accurate and a large number of annotations to achieve outstanding performance in medical image segmentation.
WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image
Deep learning-based medical image segmentation has shown the potential to reduce manual delineation efforts, but it still requires a large-scale fine annotated dataset for training, and there is a lack of large-scale datasets covering the whole abdomen region with accurate and detailed annotations for the whole abdominal organ segmentation.
Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis from Lung CT Scans with Multi-Scale Guided Dense Attention
Computed Tomography (CT) plays an important role in monitoring radiation-induced Pulmonary Fibrosis (PF), where accurate segmentation of the PF lesions is highly desired for diagnosis and treatment follow-up.
Domain Composition and Attention for Unseen-Domain Generalizable Medical Image Segmentation
First, we present a domain composition method that represents one certain domain by a linear combination of a set of basis representations (i. e., a representation bank).
Multi-frame Collaboration for Effective Endoscopic Video Polyp Detection via Spatial-Temporal Feature Transformation
For example, STFT improves the still image baseline FCOS by 10. 6% and 20. 6% on the comprehensive F1-score of the polyp localization task in CVC-Clinic and ASUMayo datasets, respectively, and outperforms the state-of-the-art video-based method by 3. 6% and 8. 0%, respectively.
Hybrid Supervision Learning for Pathology Whole Slide Image Classification
To handle this problem, we propose a hybrid supervision learning framework for this kind of high resolution images with sufficient image-level coarse annotations and a few pixel-level fine labels.
Multi-Compound Transformer for Accurate Biomedical Image Segmentation
The recent vision transformer(i. e. for image classification) learns non-local attentive interaction of different patch tokens.
Self-Ensembling Contrastive Learning for Semi-Supervised Medical Image Segmentation
In this paper, we aim to boost the performance of semi-supervised learning for medical image segmentation with limited labels using a self-ensembling contrastive learning technique.
Learning Unknown from Correlations: Graph Neural Network for Inter-novel-protein Interaction Prediction
The study of multi-type Protein-Protein Interaction (PPI) is fundamental for understanding biological processes from a systematic perspective and revealing disease mechanisms.
MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning
To solve these problems, we propose a novel deep learning-based interactive segmentation method that not only has high efficiency due to only requiring clicks as user inputs but also generalizes well to a range of previously unseen objects.
SCPM-Net: An Anchor-free 3D Lung Nodule Detection Network using Sphere Representation and Center Points Matching
To overcome these problems, we propose a 3D sphere representation-based center-points matching detection network that is anchor-free and automatically predicts the position, radius, and offset of nodules without the manual design of nodule/anchor parameters.
Automatic Segmentation of Organs-at-Risk from Head-and-Neck CT using Separable Convolutional Neural Network with Hard-Region-Weighted Loss
Despite the stateof-the-art performance achieved by Convolutional Neural Networks (CNNs) for automatic segmentation of OARs, existing methods do not provide uncertainty estimation of the segmentation results for treatment planning, and their accuracy is still limited by several factors, including the low contrast of soft tissues in CT, highly imbalanced sizes of OARs and large inter-slice spacing.
Annotation-Efficient Learning for Medical Image Segmentation based on Noisy Pseudo Labels and Adversarial Learning
In this paper, we propose an annotation-efficient learning framework for segmentation tasks that avoids annotations of training images, where we use an improved Cycle-Consistent Generative Adversarial Network (GAN) to learn from a set of unpaired medical images and auxiliary masks obtained either from a shape model or public datasets.
Multi-modal AsynDGAN: Learn From Distributed Medical Image Data without Sharing Private Information
As deep learning technologies advance, increasingly more data is necessary to generate general and robust models for various tasks.
Contrastive Learning of Relative Position Regression for One-Shot Object Localization in 3D Medical Images
To address this problem, we present a one-shot framework for organ and landmark localization in volumetric medical images, which does not need any annotation during the training stage and could be employed to locate any landmarks or organs in test images given a support (reference) image during the inference stage.
Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency
In this paper, we propose a novel framework with Uncertainty Rectified Pyramid Consistency (URPC) regularization for semi-supervised NPC GTV segmentation.
Learning Euler's Elastica Model for Medical Image Segmentation
Inspired by Euler's Elastica model and recent active contour models introduced into the field of deep learning, we propose a novel active contour with elastica (ACE) loss function incorporating Elastica (curvature and length) and region information as geometrically-natural constraints for the image segmentation tasks.
CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation
Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object.
UXNet: Searching Multi-level Feature Aggregation for 3D Medical Image Segmentation
Unlike the recent neural architecture search (NAS) methods that typically searched the optimal operators in each network layer, but missed a good strategy to search for feature aggregations, this paper proposes a novel NAS method for 3D medical image segmentation, named UXNet, which searches both the scale-wise feature aggregation strategies as well as the block-wise operators in the encoder-decoder network.
Semi-supervised Medical Image Segmentation through Dual-task Consistency
Concretely, we use a dual-task deep network that jointly predicts a pixel-wise segmentation map and a geometry-aware level set representation of the target.
Multi-organ Segmentation via Co-training Weight-averaged Models from Few-organ Datasets
In most scenarios, one might obtain annotations of a single or a few organs from one training set, and obtain annotations of the the other organs from another set of training images.
Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images
To alleviate such tedious and manual effort, in this paper we propose a novel weakly supervised segmentation framework based on partial points annotation, i. e., only a small portion of nuclei locations in each image are labeled.
Automatic Ischemic Stroke Lesion Segmentation from Computed Tomography Perfusion Images by Image Synthesis and Attention-Based Deep Neural Networks
Experimental results showed that our framework achieved the top performance on ISLES 2018 challenge and: 1) our method using synthesized pseudo DWI outperformed methods segmenting the lesion from perfusion parameter maps directly; 2) the feature extractor exploiting additional spatiotemporal CTA images led to better synthesized pseudo DWI quality and higher segmentation accuracy; and 3) the proposed loss functions and network structure improved the pseudo DWI synthesis and lesion segmentation performance.
Uncertainty-Guided Efficient Interactive Refinement of Fetal Brain Segmentation from Stacks of MRI Slices
Experimental results show that: (1) our proposed CNN obtains uncertainty estimation in real time which correlates well with mis-segmentations, (2) the proposed interactive level set is effective and efficient for refinement, (3) UGIR obtains accurate refinement results with around 30% improvement of efficiency by using uncertainty to guide user interactions.
Weakly Supervised Vessel Segmentation in X-ray Angiograms by Self-Paced Learning from Noisy Labels with Suggestive Annotation
The segmentation of coronary arteries in X-ray angiograms by convolutional neural networks (CNNs) is promising yet limited by the requirement of precisely annotating all pixels in a large number of training images, which is extremely labor-intensive especially for complex coronary trees.
SenseCare: A Research Platform for Medical Image Informatics and Interactive 3D Visualization
To this end, we have developed SenseCare research platform, which is designed to facilitate translational research on intelligent diagnosis and treatment planning in various clinical scenarios.
Human-Computer Interaction Image and Video Processing
CFEA: Collaborative Feature Ensembling Adaptation for Domain Adaptation in Unsupervised Optic Disc and Cup Segmentation
Our proposed CFEA is an interactive paradigm which presents an exquisite of collaborative adaptation through both adversarial learning and ensembling weights.
Collaborative Multi-agent Learning for MR Knee Articular Cartilage Segmentation
However, effective and efficient delineation of all the knee articular cartilages in large-sized and high-resolution 3D MR knee data is still an open challenge.
Automatic Health Problem Detection from Gait Videos Using Deep Neural Networks
This study presents a starting point toward a powerful tool for automatic classification of gait disorders and can be used as a basis for future applications of Deep Learning in clinical gait analysis.
Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding
In order to address these limitations, we present tree-structured ConvLSTM models for tree-structured image analysis tasks which can be trained end-to-end.
Residual Attention based Network for Hand Bone Age Assessment
The hierarchical attention components of the residual attention subnet force our network to focus on the key components of the X-ray images and generate the final predictions as well as the associated visual supports, which is similar to the assessment procedure of clinicians.
Quantized Densely Connected U-Nets for Efficient Landmark Localization
Finally, to reduce the memory consumption and high precision operations both in training and testing, we further quantize weights, inputs, and gradients of our localization network to low bit-width numbers.
Ranked #19 on
Pose Estimation
on MPII Human Pose
CR-GAN: Learning Complete Representations for Multi-view Generation
Generating multi-view images from a single-view input is an essential yet challenging problem.
Interactive Reinforcement Learning for Object Grounding via Self-Talking
In this paper, we introduce an interactive training method to improve the natural language conversation system for a visual grounding task.
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks
In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) aiming at generating high-resolution photo-realistic images.
Ranked #6 on
Text-to-Image Generation
on Oxford 102 Flowers
StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications.
Ranked #4 on
Text-to-Image Generation
on Oxford 102 Flowers
(Inception score metric)
Multispectral Deep Neural Networks for Pedestrian Detection
Multispectral pedestrian detection is essential for around-the-clock applications, e. g., surveillance and autonomous driving.
Ranked #5 on
Object Detection
on STCrowd
SPDA-CNN: Unifying Semantic Part Detection and Abstraction for Fine-Grained Recognition
In this paper, we propose a new CNN architecture that integrates semantic part detection and abstraction (SPDA-CNN) for fine-grained classification.
Visual Tracking via Reliable Memories
In this paper, we propose a novel visual tracking framework that intelligently discovers reliable patterns from a wide range of video to resist drift error for long-term tracking tasks.
Embedding Label Structures for Fine-Grained Feature Representation
However, previous studies have rarely focused on learning a fined-grained and structured feature representation that is able to locate similar images at different levels of relevance, e. g., discovering cars from the same make or the same model, both of which require high precision.
Learning Binary Codes for Maximum Inner Product Search
Inspired by the latest advance in asymmetric hashing schemes, we propose an asymmetric binary code learning framework based on inner product fitting.
PIEFA: Personalized Incremental and Ensemble Face Alignment
Face alignment, especially on real-time or large-scale sequential images, is a challenging task with broad applications.
Fine-Grained Histopathological Image Analysis via Robust Segmentation and Large-Scale Retrieval
Computer-aided diagnosis of medical images requires thorough analysis of image details.
