Search Results for author:
Found 74 papers, 38 papers with code
Silence is Not Consensus: Disrupting Agreement Bias in Multi-Agent LLMs via Catfish Agent for Clinical Decision Making
Large language models (LLMs) have demonstrated strong potential in clinical question answering, with recent multi-agent frameworks further improving diagnostic accuracy via collaborative reasoning.
MedITok: A Unified Tokenizer for Medical Image Synthesis and Interpretation
Advanced autoregressive models have reshaped multimodal AI.
Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery
Accurate 3D reconstruction of hands and instruments is critical for vision-based analysis of ophthalmic microsurgery, yet progress has been hampered by the lack of realistic, large-scale datasets and reliable annotation tools.
RetinaLogos: Fine-Grained Synthesis of High-Resolution Retinal Images Through Captions
The scarcity of high-quality, labelled retinal imaging data, which presents a significant challenge in the development of machine learning models for ophthalmology, hinders progress in the field.
Ophora: A Large-Scale Data-Driven Text-Guided Ophthalmic Surgical Video Generation Model
In ophthalmic surgery, developing an AI system capable of interpreting surgical videos and predicting subsequent operations requires numerous ophthalmic surgical videos with high-quality annotations, which are difficult to collect due to privacy concerns and labor consumption.
Building a Human-Verified Clinical Reasoning Dataset via a Human LLM Hybrid Pipeline for Trustworthy Medical AI
Despite strong performance in medical question-answering, the clinical adoption of Large Language Models (LLMs) is critically hampered by their opaque 'black-box' reasoning, limiting clinician trust.
AOR: Anatomical Ontology-Guided Reasoning for Medical Large Multimodal Model in Chest X-Ray Interpretation
Chest X-rays (CXRs) are the most frequently performed imaging examinations in clinical settings.
Brain Foundation Models with Hypergraph Dynamic Adapter for Brain Disease Analysis
Brain diseases, such as Alzheimer's disease and brain tumors, present profound challenges due to their complexity and societal impact.
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm.
GMAI-VL-R1: Harnessing Reinforcement Learning for Multimodal Medical Reasoning
Recent advances in general medical AI have made significant strides, but existing models often lack the reasoning capabilities needed for complex medical decision-making.
Towards Interpretable Counterfactual Generation via Multimodal Autoregression
We further introduce ProgEmu, an autoregressive model that unifies the generation of counterfactual images and textual interpretations.
ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videos
Specifically, we perform one-step denoising to convert initial noises into a clip and subsequently evaluate its long-term value, leveraging a reward model anchored by previously generated content.
MSWAL: 3D Multi-class Segmentation of Whole Abdominal Lesions Dataset
With the significantly increasing incidence and prevalence of abdominal diseases, there is a need to embrace greater use of new innovations and technology for the diagnosis and treatment of patients.
FCaS: Fine-grained Cardiac Image Synthesis based on 3D Template Conditional Diffusion Model
FCaS achieves precise cardiac structure generation using Template-guided Conditional Diffusion Model (TCDM) through bidirectional mechanisms, which provides the fine-grained topological structure information of target image through the guidance of template.
MM-Eureka: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning
We present MM-Eureka, a multimodal reasoning model that successfully extends large-scale rule-based reinforcement learning (RL) to multimodal reasoning.
Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategies
The evaluation and improvement of medical large language models (LLMs) are critical for their real-world deployment, particularly in ensuring accuracy, safety, and ethical alignment.
Multimodal Human-AI Synergy for Medical Imaging Quality Control: A Hybrid Intelligence Framework with Adaptive Dataset Curation and Closed-Loop Evaluation
To address this challenge, in this study, we establish a standardized dataset and evaluation framework for medical imaging QC, systematically assessing large language models (LLMs) in image quality assessment and report standardization.
TCM-3CEval: A Triaxial Benchmark for Assessing Responses from Large Language Models in Traditional Chinese Medicine
Large language models (LLMs) excel in various NLP tasks and modern medicine, but their evaluation in traditional Chinese medicine (TCM) is underexplored.
A Novel Ophthalmic Benchmark for Evaluating Multimodal Large Language Models with Fundus Photographs and OCT Images
These shortcomings hinder the accurate assessment of MLLMs' ability to interpret OCT scans and their broader applicability in ophthalmology.
Robust Multimodal Learning for Ophthalmic Disease Grading via Disentangled Representation
The Disentangled Representation Learning module separates multimodal data into modality-common and modality-unique representations, reducing feature entanglement and enhancing both robustness and interpretability in ophthalmic disease diagnosis.
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation
Recent multimodal large language models (MLLMs) have demonstrated significant potential in open-ended conversation, generating more accurate and personalized responses.
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Surgical practice involves complex visual interpretation, procedural skills, and advanced medical knowledge, making surgical vision-language pretraining (VLP) particularly challenging due to this complexity and the limited availability of annotated data.
GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI
Despite significant advancements in general AI, its effectiveness in the medical domain is limited by the lack of specialized medical knowledge.
SegBook: A Simple Baseline and Cookbook for Volumetric Medical Image Segmentation
To address this problem, a large-scale benchmark for comprehensive evaluation is crucial for finding these conditions.
Interactive Medical Image Segmentation: A Benchmark Dataset and Baseline
To facilitate research on foundational models in medical computer vision, we release the IMed-361M and model at https://github. com/uni-medical/IMIS-Bench.
Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?
We are committed to expanding this benchmark to encourage more innovation of AI algorithms for the medical domain.
Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance
Multimodal large language models (MLLMs) have demonstrated impressive performance in vision-language tasks across a broad spectrum of domains.
SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding
We will fully release SlideChat, SlideInstruction and SlideBench as open-source resources to facilitate research and development in computational pathology.
PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgery
The field of computer vision applied to videos of minimally invasive surgery is ever-growing.
A Survey for Large Language Models in Biomedicine
To address these challenges, we also identify future research directions of LLM in biomedicine including federated learning methods to preserve data privacy and integrating explainable AI methodologies to enhance the transparency of LLMs.
TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineering
While large language models (LLMs) have achieved much progress in the domain of natural language processing, their potential in protein engineering remains largely unexplored.
GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI
Large Vision-Language Models (LVLMs) are capable of handling diverse data types such as imaging, text, and physiological signals, and can be applied in various fields.
Multi-modal MRI Translation via Evidential Regression and Distribution Calibration
Multi-modal Magnetic Resonance Imaging (MRI) translation leverages information from source MRI sequences to generate target modalities, enabling comprehensive diagnosis while overcoming the limitations of acquiring all sequences.
SAM-Med3D-MoE: Towards a Non-Forgetting Segment Anything Model via Mixture of Experts for 3D Medical Image Segmentation
Supervised Finetuning (SFT) serves as an effective way to adapt such foundation models for task-specific downstream tasks but at the cost of degrading the general knowledge previously stored in the original foundation model. To address this, we propose SAM-Med3D-MoE, a novel framework that seamlessly integrates task-specific finetuned models with the foundational model, creating a unified model at minimal additional training expense for an extra gating network.
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text
In this paper, we introduce OmniCorpus, a 10 billion-scale image-text interleaved dataset.
A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understanding
The parallels between protein sequences and natural language in their sequential structures have inspired the application of large language models (LLMs) to protein understanding.
OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine
The emerging trend of advancing generalist artificial intelligence, such as GPTv4 and Gemini, has reshaped the landscape of research (academia and industry) in machine learning and many other research areas.
OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM
Importantly, all images in this benchmark are sourced from authentic medical scenarios, ensuring alignment with the requirements of the medical field and suitability for evaluating LVLMs.
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
We propose SPHINX-X, an extensive Multimodality Large Language Model (MLLM) series developed upon SPHINX.
Ranked #14 on
Video Question Answering
on MVBench
Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models
Based on this observation we generalize diffusion methods to model font generative process by separating the reverse diffusion process into three stages with different functions: The structure construction stage first generates the structure information for the target character based on the source image and the font transfer stage subsequently transforms the source font to the target font.
Towards the Unification of Generative and Discriminative Visual Foundation Model: A Survey
The advent of foundation models, which are pre-trained on vast datasets, has ushered in a new era of computer vision, characterized by their robustness and remarkable zero-shot generalization capabilities.
SegRap2023: A Benchmark of Organs-at-Risk and Gross Tumor Volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma
The precise delineation of Gross Tumor Volumes (GTVs) and Organs-At-Risk (OARs) is crucial in radiation treatment, directly impacting patient prognosis.
Prompted Contextual Transformer for Incomplete-View CT Reconstruction
To enjoy the multi-setting synergy in a single model, we propose a novel Prompted Contextual Transformer (ProCT) for incomplete-view CT reconstruction.
Enhancing Medical Task Performance in GPT-4V: A Comprehensive Study on Prompt Engineering Strategies
OpenAI's latest large vision-language model (LVLM), GPT-4V(ision), has piqued considerable interest for its potential in medical applications.
SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million masks
Segment Anything Model (SAM) has achieved impressive results for natural image segmentation with input prompts such as points and bounding boxes.
SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images
In this paper, we introduce SAM-Med3D for general-purpose segmentation on volumetric medical images.
A-Eval: A Benchmark for Cross-Dataset Evaluation of Abdominal Multi-Organ Segmentation
To address these questions, we introduce A-Eval, a benchmark for the cross-dataset Evaluation ('Eval') of Abdominal ('A') multi-organ segmentation.
Artifact Restoration in Histology Images with Diffusion Probabilistic Models
Histological whole slide images (WSIs) can be usually compromised by artifacts, such as tissue folding and bubbles, which will increase the examination difficulty for both pathologists and Computer-Aided Diagnosis (CAD) systems.
Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation
To make up for its deficiency when applying transfer learning to medical image segmentation, in this paper, we therefore propose a new Transferability Estimation (TE) method.
MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification
Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications.
STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training
However, the state-of-the-art models for medical image segmentation are still small-scale, with their parameters only in the tens of millions.
Learning with Explicit Shape Priors for Medical Image Segmentation
Hence, in our work, we proposed a novel shape prior module (SPM), which can explicitly introduce shape priors to promote the segmentation performance of UNet-based models.
Token Sparsification for Faster Medical Image Segmentation
To this end, we reformulate segmentation as a sparse encoding -> token completion -> dense decoding (SCD) pipeline.
Generative Model Based Noise Robust Training for Unsupervised Domain Adaptation
D-CFA minimizes the domain gap by augmenting the source data with distribution-sampled target features, and trains a noise-robust discriminative classifier by using target domain knowledge from the generative models.
FCN+: Global Receptive Convolution Makes FCN Great Again
The motivation of GRC is that different channels of a convolutional filter can have different grid sampling locations across the whole input feature map.
Neural Transformation Fields for Arbitrary-Styled Font Generation
Few-shot font generation (FFG), aiming at generating font images with a few samples, is an emerging topic in recent years due to the academic and commercial values.
Exploring Vanilla U-Net for Lesion Segmentation from Whole-body FDG-PET/CT Scans
The combination of PET-based metabolic and CT-based anatomic information can contribute to better tumor segmentation results.
An evaluation of U-Net in Renal Structure Segmentation
Renal structure segmentation from computed tomography angiography~(CTA) is essential for many computer-assisted renal cancer treatment applications.
Vision Transformer Adapter for Dense Predictions
This work investigates a simple yet powerful dense prediction task adapter for Vision Transformer (ViT).
Ranked #5 on
Semantic Segmentation
on PASCAL Context
StructToken : Rethinking Semantic Segmentation with Structural Prior
In previous deep-learning-based methods, semantic segmentation has been regarded as a static or dynamic per-pixel classification task, \textit{i. e.,} classify each pixel representation to a specific category.
Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentation
Masked Autoencoder (MAE) has recently been shown to be effective in pre-training Vision Transformers (ViT) for natural image analysis.
Dynamic Instance Domain Adaptation
In this paper, we address both single-source and multi-source UDA from a completely different perspective, which is to view each instance as a fine domain.
OpenKBP-Opt: An international and reproducible evaluation of 76 knowledge-based planning pipelines
The dose predictions were input to four optimization models to form 76 unique KBP pipelines that generated 7600 plans.
Group Shift Pointwise Convolution for Volumetric Medical Image Segmentation
Recent studies have witnessed the effectiveness of 3D convolutions on segmenting volumetric medical images.
A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images
An essential step of RT planning is the accurate segmentation of various organs-at-risks (OARs) in HaN CT images.
Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition
To this end, we propose an Attention-Driven Dynamic Graph Convolutional Network (ADD-GCN) to dynamically generate a specific graph for each image.
Ranked #24 on
Multi-Label Classification
on MS-COCO
MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response
Furthermore, we experiment the proposed method on an in-house, retrospective dataset of real-world non-small cell lung cancer patients under anti-PD-1 immunotherapy.
Ranked #1 on
Text-To-Speech Synthesis
on 20000 utterances
(using extra training data)
EfficientFCN: Holistically-guided Decoding for Semantic Segmentation
State-of-the-art semantic segmentation algorithms are mostly based on dilated Fully Convolutional Networks (dilatedFCN), which adopt dilated convolutions in the backbone networks to extract high-resolution feature maps for achieving high-performance segmentation performance.
Tensor Low-Rank Reconstruction for Semantic Segmentation
Then we use these rank-1 tensors to recover the high-rank context features through our proposed tensor reconstruction module (TRM).
Learning to Predict Context-adaptive Convolution for Semantic Segmentation
Long-range contextual information is essential for achieving high-performance semantic segmentation.
Dynamic Multi-Scale Filters for Semantic Segmentation
DMNet is composed of multiple Dynamic Convolutional Modules (DCMs) arranged in parallel, each of which exploits context-aware filters to estimate semantic representation for a specific scale.
Adaptive Pyramid Context Network for Semantic Segmentation
Recent studies witnessed that context features can significantly improve the performance of deep semantic segmentation networks.
Ranked #41 on
Thermal Image Segmentation
on MFN Dataset
Prostate Segmentation using 2D Bridged U-net
To address the aforementioned three problems, we propose and validate a deeper network that can fit medical image datasets that are usually small in the sample size.
