Search Results for author:
Found 91 papers, 53 papers with code
A Survey on Latent Reasoning
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps.
FedVLMBench: Benchmarking Federated Fine-Tuning of Vision-Language Models
Vision-Language Models (VLMs) have demonstrated remarkable capabilities in cross-modal understanding and generation by integrating visual and textual information.
More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
Test-time compute has empowered multimodal large language models to generate extended reasoning chains, yielding strong performance on tasks such as multimodal math reasoning.
MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning
To build MedFrameQA both at scale and in high-quality, we develop 1) an automated pipeline that extracts temporally coherent frames from medical videos and constructs VQA items whose content evolves logically across images, and 2) a multiple-stage filtering strategy, including model-based and manual review, to preserve data clarity, difficulty, and medical relevance.
ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoning
Federated Learning (FL) has emerged as a promising paradigm for collaborative model training while preserving data privacy across decentralized participants.
$\texttt{Complex-Edit}$: CoT-Like Instruction Generation for Complexity-Controllable Image Editing Benchmark
We introduce $\texttt{Complex-Edit}$, a comprehensive benchmark designed to systematically evaluate instruction-based image editing models across instructions of varying complexity.
SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
This work revisits the dominant supervised fine-tuning (SFT) then reinforcement learning (RL) paradigm for training Large Vision-Language Models (LVLMs), and reveals a key finding: SFT can significantly undermine subsequent RL by inducing ``pseudo reasoning paths'' imitated from expert models.
MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
Our pipeline generates detailed reasoning for various medical questions from 7 medical datasets, resulting in a dataset of 32, 682 question-answer pairs, each with detailed, step-by-step explanations.
Exploring the Vulnerabilities of Federated Learning: A Deep Dive into Gradient Inversion Attacks
To fill this gap, we first undertake a systematic review of GIA and categorize existing methods into three types, i. e., \textit{optimization-based} GIA (OP-GIA), \textit{generation-based} GIA (GEN-GIA), and \textit{analytics-based} GIA (ANA-GIA).
Multi-Class Segmentation of Aortic Branches and Zones in Computed Tomography Angiography: The AortaSeg24 Challenge
Furthermore, no open-source dataset is currently available to support the development of multi-class aortic segmentation methods.
UD-Mamba: A pixel-level uncertainty-driven Mamba model for medical image segmentation
Recent advancements have highlighted the Mamba framework, a state-space model known for its efficiency in capturing long-range dependencies with linear computational complexity.
ARFlow: Autogressive Flow with Hybrid Linear Attention
Flow models are effective at progressively generating realistic images, but they generally struggle to capture long-range dependencies during the generation process as they compress all the information from previous time steps into a single corrupted image.
Humanity's Last Exam
However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities.
Ranked #4 on
Humanity's Last Exam
on Humanity's Last Exam
Mamba-Reg: Vision Mamba Also Needs Registers
Qualitative observations suggest, compared to vanilla Vision Mamba, MambaReg's feature maps appear cleaner and more focused on semantically meaningful regions.
Adventurer: Optimizing Vision Mamba Architecture Designs for Efficiency
In this work, we introduce the Adventurer series models where we treat images as sequences of patch tokens and employ uni-directional language models to learn visual representations.
Efficient MedSAMs: Segment Anything in Medical Images on Laptop
Promptable segmentation foundation models have emerged as a transformative approach to addressing the diverse needs in medical images, but most existing models require expensive computing, posing a big barrier to their adoption in clinical practice.
A New Federated Learning Framework Against Gradient Inversion Attacks
By revisiting the key to privacy exposure in FL under GIA, which lies in the frequent sharing of model gradients that contain private data, we take a new perspective by designing a novel privacy preserve FL framework that effectively ``breaks the direct connection'' between the shared parameters and the local private data to defend against GIA.
Generative Image Layer Decomposition with Visual Effects
Layered representations, which allow for independent editing of image components, are essential for user-driven content creation, yet existing approaches often struggle to decompose image into plausible layers with accurately retained transparent visual effects such as shadows and reflections.
Causal Image Modeling for Efficient Visual Understanding
In this work, we present a comprehensive analysis of causal image modeling and introduce the Adventurer series models where we treat images as sequences of patch tokens and employ uni-directional language models to learn visual representations.
Story-Adapter: A Training-free Iterative Framework for Long Story Visualization
Specifically, we propose an iterative paradigm to refine each generated image, leveraging both the text prompt and all generated images from the previous iteration.
A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?
Large language models (LLMs) have exhibited remarkable capabilities across various domains and tasks, pushing the boundaries of our knowledge in learning and cognition.
Tackling Data Heterogeneity in Federated Learning via Loss Decomposition
To mitigate the impact of data heterogeneity on FL performance, we start with analyzing how FL training influence FL performance by decomposing the global loss into three terms: local loss, distribution shift loss and aggregation loss.
MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine
Unlike the existing multimodal datasets, which are limited by the availability of image-text pairs, we have developed the first automated pipeline that scales up multimodal data by generating multigranular visual and textual annotations in the form of image-ROI-description triplets without the need for any paired text descriptions.
Restorer: Removing Multi-Degradation with All-Axis Attention and Prompt Guidance
There are many excellent solutions in image restoration. However, most methods require on training separate models to restore images with different types of degradation. Although existing all-in-one models effectively address multiple types of degradation simultaneously, their performance in real-world scenarios is still constrained by the task confusion problem. In this work, we attempt to address this issue by introducing \textbf{Restorer}, a novel Transformer-based all-in-one image restoration model. To effectively address the complex degradation present in real-world images, we propose All-Axis Attention (AAA), a mechanism that simultaneously models long-range dependencies across both spatial and channel dimensions, capturing potential correlations along all axes. Additionally, we introduce textual prompts in Restorer to incorporate explicit task priors, enabling the removal of specific degradation types based on user instructions.
DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor
In this paper, we present Deep Degradation Response (DDR), a method to quantify changes in image deep features under varying degradation conditions.
What If We Recaption Billions of Web Images with LLaMA-3?
For discriminative models like CLIP, we observe enhanced zero-shot performance in cross-modal retrieval tasks.
Ranked #127 on
Visual Question Answering
on MM-Vet
Medical Vision Generalist: Unifying Medical Imaging Tasks in Context
This study presents Medical Vision Generalist (MVG), the first foundation model capable of handling various medical imaging tasks -- such as cross-modal synthesis, image segmentation, denoising, and inpainting -- within a unified image-to-image generation framework.
Scaling White-Box Transformers for Vision
CRATE, a white-box transformer architecture designed to learn compressed and sparse representations, offers an intriguing alternative to standard vision transformers (ViTs) due to its inherent mathematical interpretability.
Mamba-R: Vision Mamba ALSO Needs Registers
Similar to Vision Transformers, this paper identifies artifacts also present within the feature maps of Vision Mamba.
Fast-DDPM: Fast Denoising Diffusion Probabilistic Models for Medical Image-to-Image Generation
This is primarily due to the high computational cost associated with (1) the use of large number of time steps (e. g., 1, 000) in diffusion processes and (2) the increased dimensionality of medical images, which are often 3D or 4D.
A Flexible 2.5D Medical Image Segmentation Approach with In-Slice and Cross-Slice Attention
This module uses the cross-slice attention mechanism to effectively capture 3D spatial information by learning long-range dependencies between the center slice (for segmentation) and its neighboring slices.
RetinaRegNet: A Zero-Shot Approach for Retinal Image Registration
We introduce RetinaRegNet, a zero-shot image registration model designed to register retinal images with minimal overlap, large deformations, and varying image quality.
HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing
This study introduces HQ-Edit, a high-quality instruction-based image editing dataset with around 200, 000 edits.
Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding
This paper introduces H-SAM: a prompt-free adaptation of SAM tailored for efficient fine-tuning of medical images via a two-stage hierarchical decoding procedure.
3D-TransUNet for Brain Metastases Segmentation in the BraTS2023 Challenge
We identify that the Decoder-only 3D-TransUNet model should offer enhanced efficacy in the segmentation of brain metastases, as indicated by our 5-fold cross-validation on the training set.
MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections
This strategy enriches the diffusion process with structured 3D information, enhancing detail and reducing noise in localized 2D images.
CIS-UNet: Multi-Class Segmentation of the Aorta in Computed Tomography Angiography via Context-Aware Shifted Window Self-Attention
Advancements in medical imaging and endovascular grafting have facilitated minimally invasive treatments for aortic diseases.
FLHetBench: Benchmarking Device and State Heterogeneity in Federated Learning
Federated learning (FL) is a powerful technology that enables collaborative training of machine learning models without sharing private data among clients.
A Semantic Space is Worth 256 Language Descriptions: Make Stronger Segmentation Models with Descriptive Properties
Instead of relying solely on category-specific annotations, ProLab uses descriptive properties grounded in common sense knowledge for supervising segmentation models.
Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-training
Contrastive learning has emerged as a promising paradigm for 3D open-world understanding, i. e., aligning point cloud representation to image and text embedding space individually.
Ranked #1 on
Zero-shot 3D classification
on Objaverse LVIS
(using extra training data)
3D TransUNet: Advancing Medical Image Segmentation through Vision Transformers
In this paper, we extend the 2D TransUNet architecture to a 3D network by building upon the state-of-the-art nnU-Net architecture, and fully exploring Transformers' potential in both the encoder and decoder design.
FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning
Federated learning (FL) is an emerging paradigm in machine learning, where a shared model is collaboratively learned using data from multiple devices to mitigate the risk of data leakage.
Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Prompts
Image synthesis approaches, e. g., generative adversarial networks, have been popular as a form of data augmentation in medical image analysis tasks.
SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation
To address this limitation, we present Masked Multi-view with Swin Transformers (SwinMM), a novel multi-view pipeline for enabling accurate and data-efficient self-supervised medical image analysis.
Consistency-guided Meta-Learning for Bootstrapping Semi-Supervised Medical Image Segmentation
Specifically, our approach first involves training a segmentation model on a small set of clean labeled images to generate initial labels for unlabeled data.
Image Registration of In Vivo Micro-Ultrasound and Ex Vivo Pseudo-Whole Mount Histopathology Images of the Prostate: A Proof-of-Concept Study
Our pipeline begins with the reconstruction of pseudo-whole-mount histopathology images and a 3-dimensional (3D) micro-US volume.
MicroSegNet: A Deep Learning Approach for Prostate Segmentation on Micro-Ultrasound Images
During the training process, MicroSegNet focuses more on regions that are hard to segment (hard regions), characterized by discrepancies between expert and non-expert annotations.
BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks
Traditional biomedical artificial intelligence (AI) models, designed for specific tasks or modalities, often exhibit limited flexibility in real-world deployment and struggle to utilize holistic information.
Ranked #1 on
Text Summarization
on MeQSum
Distribution Aligned Diffusion and Prototype-guided network for Unsupervised Domain Adaptive Segmentation
In order to explore its potential further, we have taken a step forward and considered a more complex scenario in the medical image domain, specifically, under an unsupervised adaptation condition.
Unleashing the Power of Visual Prompting At the Pixel Level
This paper presents a simple and effective visual prompting method for adapting pre-trained models to downstream recognition tasks.
Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning
In this paper, we present a novel Multi-Granularity Cross-modal Alignment (MGCA) framework for generalized medical visual representation learning by harnessing the naturally exhibited semantic correspondences between medical image and radiology reports at three different levels, i. e., pathological region-level, instance-level, and disease-level.
Bag of Tricks for FGSM Adversarial Training
Adversarial training (AT) with samples generated by Fast Gradient Sign Method (FGSM), also known as FGSM-AT, is a computationally simple method to train robust networks.
Masked Autoencoders Enable Efficient Knowledge Distillers
For example, by distilling the knowledge from an MAE pre-trained ViT-L into a ViT-B, our method achieves 84. 0% ImageNet top-1 accuracy, outperforming the baseline of directly distilling a fine-tuned ViT-L by 1. 2%.
Multiple Instance Neuroimage Transformer
As a proof-of-concept, we evaluate the efficacy of our model by training it to identify sex from T1w-MRIs of two public datasets: Adolescent Brain Cognitive Development (ABCD) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA).
A Simple Data Mixing Prior for Improving Self-Supervised Learning
More notably, our SDMP is the first method that successfully leverages data mixing to improve (rather than hurt) the performance of Vision Transformers in the self-supervised setting.
Can CNNs Be More Robust Than Transformers?
The recent success of Vision Transformers is shaking the long dominance of Convolutional Neural Networks (CNNs) in image recognition for a decade.
Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging
The collection and curation of large-scale medical datasets from multiple institutions is essential for training accurate deep learning models, but privacy concerns often hinder data sharing.
In Defense of Image Pre-Training for Spatiotemporal Recognition
Inspired by this observation, we hypothesize that the key to effectively leveraging image pre-training lies in the decomposition of learning spatial and temporal features, and revisiting image pre-training as the appearance prior to initializing 3D kernels.
CD$^2$-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to collaboratively learn a shared global model.
L2B: Learning to Bootstrap Robust Models for Combating Label Noise
Extensive experiments demonstrate that our method effectively mitigates the challenges of noisy labels, often necessitating few to no validation samples, and is well generalized to other tasks such as image segmentation.
Ranked #8 on
Image Classification
on Clothing1M (using clean data)
(using extra training data)
External Attention Assisted Multi-Phase Splenic Vascular Injury Segmentation with Limited Data
The spleen is one of the most commonly injured solid organs in blunt abdominal trauma.
CD2-pFed: Cyclic Distillation-Guided Channel Decoupling for Model Personalization in Federated Learning
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to collaboratively learn a shared global model.
A collection of the accepted abstracts for the Machine Learning for Health (ML4H) symposium 2021
A collection of the accepted abstracts for the Machine Learning for Health (ML4H) symposium 2021.
RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR
Despite the routine use of electronic health record (EHR) data by radiologists to contextualize clinical history and inform image interpretation, the majority of deep learning architectures for medical imaging are unimodal, i. e., they only learn features from pixel-level information.
Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning
Federated learning is an emerging research paradigm enabling collaborative training of machine learning models among different organizations while keeping data private at each institution.
Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction
We instantiate both the global and the local classifiers by multiple instance learning (MIL), where the attention guidance, indicating roughly where the PDAC regions are, is the key to bridging them: For global MIL based normal/PDAC classification, attention serves as a weight for each instance (voxel) during MIL pooling, which eliminates the distraction from the background; For local MIL based semi-supervised PDAC segmentation, the attention guidance is inductive, which not only provides bag-level pseudo-labels to training data without per-voxel annotations for MIL training, but also acts as a proxy of an instance-level classifier.
CateNorm: Categorical Normalization for Robust Medical Image Segmentation
We propose a new normalization strategy, named categorical normalization (CateNorm), to normalize the activations according to categorical statistics.
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Medical image segmentation is an essential prerequisite for developing healthcare systems, especially for disease diagnosis and treatment planning.
Ranked #6 on
Medical Image Segmentation
on ACDC
Can Temporal Information Help with Contrastive Self-Supervised Learning?
To this end, we present Temporal-aware Contrastive self-supervised learningTaCo, as a general paradigm to enhance video CSL.
Volumetric Medical Image Segmentation: A 3D Deep Coarse-to-fine Framework and Its Adversarial Examples
Although deep neural networks have been a dominant method for many 2D vision tasks, it is still challenging to apply them to 3D tasks, such as medical image segmentation, due to the limited amount of annotated 3D data and limited computational resources.
Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation
In this paper, we present a novel unsupervised domain adaptation (UDA) method, named Domain Adaptive Relational Reasoning (DARR), to generalize 3D multi-organ segmentation models to medical data collected from different scanners and/or protocols (domains).
Neural Architecture Search for Lightweight Non-Local Networks
However, it has been rarely explored to embed the NL blocks in mobile neural networks, mainly due to the following challenges: 1) NL blocks generally have heavy computation cost which makes it difficult to be applied in applications where computational resources are limited, and 2) it is an open problem to discover an optimal configuration to embed NL blocks into mobile neural networks.
Ranked #60 on
Neural Architecture Search
on ImageNet
CAKES: Channel-wise Automatic KErnel Shrinking for Efficient 3D Networks
3D Convolution Neural Networks (CNNs) have been widely applied to 3D scene understanding, such as video analysis and volumetric image recognition.
Detecting Pancreatic Ductal Adenocarcinoma in Multi-phase CT Scans via Alignment Ensemble
Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal cancers among the population.
Deep Distance Transform for Tubular Structure Segmentation in CT Scans
Tubular structure segmentation in medical images, e. g., segmenting vessels in CT scans, serves as a vital step in the use of computers to aid in screening early stages of related diseases.
Universal Physical Camouflage Attacks on Object Detectors
In this paper, we study physical adversarial attacks on object detectors in the wild.
Hyper-Pairing Network for Multi-Phase Pancreatic Ductal Adenocarcinoma Segmentation
Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal cancers with an overall five-year survival rate of 8%.
FusionNet: Incorporating Shape and Texture for Abnormality Detection in 3D Abdominal CT Scans
Second, a FusionNet is proposed to take both the binary mask and CT image as input and perform a binary classification.
Multi-Scale Attentional Network for Multi-Focal Segmentation of Active Bleed after Pelvic Fractures
Trauma is the worldwide leading cause of death and disability in those younger than 45 years, and pelvic fractures are a major source of morbidity and mortality.
Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
Accurate multi-organ abdominal CT segmentation is essential to many clinical applications such as computer-aided intervention.
Adversarial Metric Attack and Defense for Person Re-identification
However, our work observes the extreme vulnerability of existing distance metrics to adversarial examples, generated by simply adding human-imperceptible perturbations to person images.
Learning Transferable Adversarial Examples via Ghost Networks
The critical principle of ghost networks is to apply feature-level perturbations to an existing model to potentially create a huge set of diverse models.
Abdominal multi-organ segmentation with organ-attention networks and statistical fusion
To address these challenges, we introduce a novel framework for multi-organ segmentation by using organ-attention networks with reverse connections (OAN-RCs) which are applied to 2D views, of the 3D CT volume, and output estimates which are combined by statistical fusion exploiting structural similarity.
Semi-Supervised Multi-Organ Segmentation via Deep Multi-Planar Co-Training
In multi-organ segmentation of abdominal CT scans, most existing fully supervised deep learning algorithms require lots of voxel-wise annotations, which are usually difficult, expensive, and slow to obtain.
Training Multi-organ Segmentation Networks with Sample Selection by Relaxed Upper Confident Bound
Based on the fact that very hard samples might have annotation errors, we propose a new sample selection policy, named Relaxed Upper Confident Bound (RUCB).
Improving Transferability of Adversarial Examples with Input Diversity
We hope that our proposed attack strategy can serve as a strong benchmark baseline for evaluating the robustness of networks to adversaries and the effectiveness of different defense methods in the future.
Visual Concepts and Compositional Voting
We use clustering algorithms to study the population activities of the features and extract a set of visual concepts which we show are visually tight and correspond to semantic parts of vehicles.
Recurrent Saliency Transformation Network: Incorporating Multi-Stage Visual Cues for Small Organ Segmentation
The key innovation is a saliency transformation module, which repeatedly converts the segmentation probability map from the previous iteration as spatial weights and applies these weights to the current iteration.
Ranked #1 on
Pancreas Segmentation
on TCIA Pancreas-CT Dataset
Deep Supervision for Pancreatic Cyst Segmentation in Abdominal CT Scans
Inspired by the high relevance between the location of a pancreas and its cystic region, we introduce extra deep supervision into the segmentation network, so that cyst segmentation can be improved with the help of relatively easier pancreas segmentation.
Adversarial Examples for Semantic Segmentation and Object Detection
Our observation is that both segmentation and detection are based on classifying multiple targets on an image (e. g., the basic target is a pixel or a receptive field in segmentation, and an object proposal in detection), which inspires us to optimize a loss function over a set of pixels/proposals for generating adversarial perturbations.
A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans
Deep neural networks have been widely adopted for automatic organ segmentation from abdominal CT scans.
