Search Results for author:
Found 94 papers, 51 papers with code
MedForge: Building Medical Foundation Models Like Open Source Software Development
In this study, we propose Medical Foundation Models Merging (MedForge), a cooperative framework enabling a community-driven medical foundation model development, meanwhile preventing the information leakage of raw patient data and mitigating synchronization model development issues across clinical institutions.
Improved Training Technique for Latent Consistency Models
With these strategies, we successfully train latent consistency models capable of high-quality sampling with one or two steps, significantly narrowing the performance gap between latent consistency and diffusion models.
RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment
Automated chest radiographs interpretation requires both accurate disease classification and detailed radiology report generation, presenting a significant challenge in the clinical workflow.
Rate-My-LoRA: Efficient and Adaptive Federated Model Tuning for Cardiac MRI Segmentation
In this paper, we propose a novel efficient and adaptive federate learning method for cardiac segmentation that improves model performance while reducing the bandwidth requirement.
Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Text-to-Image Generation
Flow matching has emerged as a promising framework for training generative models, demonstrating impressive empirical performance while offering relative ease of training compared to diffusion-based models.
VerSe: Integrating Multiple Queries as Prompts for Versatile Cardiac MRI Segmentation
VerSe supports both fully automatic segmentation, through object queries, and interactive mask refinement, by providing click queries when needed.
SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device
We have witnessed the unprecedented success of diffusion-based video generation over the past year.
Learning Volumetric Neural Deformable Models to Recover 3D Regional Heart Wall Motion from Multi-Planar Tagged MRI
To learn such global and local deformation for 2D apparent motion mapping to 3D true motion, we design a hybrid point transformer, which incorporates both point cross-attention and self-attention mechanisms.
DiMSUM: Diffusion Mamba -- A Scalable and Unified Spatial-Frequency Method for Image Generation
We introduce a novel state-space architecture for diffusion models, effectively harnessing spatial and frequency information to enhance the inductive bias towards local features in input images for image generation tasks.
Continuous Spatio-Temporal Memory Networks for 4D Cardiac Cine MRI Segmentation
Current cardiac cine magnetic resonance image (cMR) studies focus on the end diastole (ED) and end systole (ES) phases, while ignoring the abundant temporal information in the whole image sequence.
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models
Discrete diffusion models have achieved success in tasks like image generation and masked language modeling but face limitations in controlled content editing.
Resolving Inconsistent Semantics in Multi-Dataset Image Segmentation
Notably, on four benchmark datasets with label space inconsistencies during inference, we outperform previous methods by 1. 6% mIoU for semantic segmentation, 9. 1% PQ for panoptic segmentation, 12. 1% AP for instance segmentation, and 3. 0% in the newly proposed PIQ metric.
New Capability to Look Up an ASL Sign from a Video Example
The user submits a video for analysis and is presented with the five most likely sign matches, in decreasing order of likelihood, so that the user can confirm the selection and then be taken to our ASLLRP Sign Bank entry for that sign.
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
Large Language Models (LLMs) often suffer from overconfidence during inference, particularly when adapted to downstream domain-specific tasks with limited data.
Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification
Although explainability is essential in the clinical diagnosis, most deep learning models still function as black boxes without elucidating their decision-making process.
Score-Guided Diffusion for 3D Human Recovery
We present Score-Guided Human Mesh Recovery (ScoreHMR), an approach for solving inverse problems for 3D human pose and shape reconstruction.
AVID: Any-Length Video Inpainting with Diffusion Model
Given a video, a masked region at its initial frame, and an editing prompt, it requires a model to do infilling at each frame following the editing guidance while keeping the out-of-mask region intact.
Improving Compositional Text-to-image Generation with Large Vision-Language Models
However, compositional text-to-image models frequently encounter difficulties in generating high-quality images that accurately align with input texts describing multiple objects, variable attributes, and intricate spatial relationships.
Neural Deformable Models for 3D Bi-Ventricular Heart Shape Reconstruction and Modeling from 2D Sparse Cardiac Magnetic Resonance Imaging
We model the bi-ventricular shape using blended deformable superquadrics, which are parameterized by a set of geometric parameter functions and are capable of deforming globally and locally.
On the Challenges and Perspectives of Foundation Models for Medical Image Analysis
This article discusses the opportunities, applications and future directions of large-scale pre-trained models, i. e., foundation models, for analyzing medical images.
Improving Tuning-Free Real Image Editing with Proximal Guidance
Null-text inversion (NTI) optimizes null embeddings to align the reconstruction and inversion trajectories with larger CFG scales, enabling real image editing with cross-attention control.
Learning Articulated Shape with Keypoint Pseudo-labels from Web Images
It is based on two key insights: (1) 2D keypoint estimation networks trained on as few as 50-150 images of a given object category generalize well and generate reliable pseudo-labels; (2) a data selection mechanism can automatically create a "curated" subset of the unlabeled web images that can be used for training -- we evaluate four data selection methods.
OmniLabel: A Challenging Benchmark for Language-Based Object Detection
With more than 28K unique object descriptions on over 25K images, OmniLabel provides a challenging benchmark with diverse and complex object descriptions in a naturally open-vocabulary setting.
Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies
Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned.
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning
Diffusion models have achieved remarkable success in text-to-image generation, enabling the creation of high-quality images from text prompts or other modalities.
SINE: SINgle Image Editing with Text-to-Image Diffusion Models
We propose a novel model-based guidance built upon the classifier-free guidance so that the knowledge from the model trained on a single image can be distilled into the pre-trained diffusion model, enabling content creation even with one given image.
Diffusion Guided Domain Adaptation of Image Generators
Can a text-to-image diffusion model be used as a training objective for adapting a GAN generator to another domain?
Automatic Tooth Segmentation from 3D Dental Model using Deep Learning: A Quantitative Analysis of what can be learnt from a Single 3D Dental Model
We conclude that the segmentation methods can learn a great deal of information from a single 3D tooth point cloud scan under suitable conditions e. g. data augmentation.
Exploiting Unlabeled Data with Vision and Language Models for Object Detection
We propose a novel method that leverages the rich semantics available in recent vision and language models to localize and classify objects in unlabeled images, effectively generating pseudo labels for object detection.
Ranked #20 on
Open Vocabulary Object Detection
on MSCOCO
A Dynamic Data Driven Approach for Explainable Scene Understanding
Suppose that an agent utilizing one or more sensors is placed in an unknown environment, and based on its sensory input, the agent needs to assign some label to the perceived scene.
DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Method
Joint 2D cardiac segmentation and 3D volume reconstruction are fundamental to building statistical cardiac anatomy models and understanding functional mechanisms from motion patterns.
A Manifold View of Adversarial Risk
Assuming data lies in a manifold, we investigate two new types of adversarial risk, the normal adversarial risk due to perturbation along normal direction, and the in-manifold adversarial risk due to perturbation within the manifold.
TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformers
Combining information from multi-view images is crucial to improve the performance and robustness of automated methods for disease diagnosis.
Global Matching with Overlapping Attention for Optical Flow Estimation
In this paper, inspired by the traditional matching-optimization methods where matching is introduced to handle large displacements before energy-based optimizations, we introduce a simple but effective global matching step before the direct regression and develop a learning-based matching-optimization framework, namely GMFlowNet.
Ranked #5 on
Optical Flow Estimation
on KITTI 2015
Region Proposal Rectification Towards Robust Instance Segmentation of Biological Images
Top-down instance segmentation framework has shown its superiority in object detection compared to the bottom-up framework.
Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning
In addition, our model can extract visual information as suggested by the text prompt, e. g., "an object in image one is moving northeast", and generate corresponding videos.
Modality Bank: Learn multi-modality images across data centers without sharing medical data
Multi-modality images have been widely used and provide comprehensive information for medical image analysis.
Contrastive and Selective Hidden Embeddings for Medical Image Segmentation
Medical image segmentation has been widely recognized as a pivot procedure for clinical diagnosis, analysis, and treatment planning.
ASL Video Corpora & Sign Bank: Resources Available through the American Sign Language Linguistic Research Project (ASLLRP)
These data have been used for many types of research in linguistics and in computer-based sign language recognition from video; examples of such research are provided in the latter part of this article.
AE-StyleGAN: Improved Training of Style-Based Auto-Encoders
StyleGANs have shown impressive results on data generation and manipulation in recent years, thanks to its disentangled style latent space.
Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis from Lung CT Scans with Multi-Scale Guided Dense Attention
Computed Tomography (CT) plays an important role in monitoring radiation-induced Pulmonary Fibrosis (PF), where accurate segmentation of the PF lesions is highly desired for diagnosis and treatment follow-up.
Global and Local Interpretation of black-box Machine Learning models to determine prognostic factors from early COVID-19 data
We explore one of the most recent techniques called symbolic metamodeling to find the mathematical expression of the machine learning models for COVID-19.
Dual Projection Generative Adversarial Networks for Conditional Image Generation
We then propose an improved cGAN model with Auxiliary Classification that directly aligns the fake and real conditionals $P(\text{class}|\text{image})$ by minimizing their $f$-divergence.
UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation
In this study, we present UTNet, a simple yet powerful hybrid Transformer architecture that integrates self-attention into a convolutional neural network for enhancing medical image segmentation.
Enabling Data Diversity: Efficient Automatic Augmentation via Regularized Adversarial Training
Data augmentation has proved extremely useful by increasing training data variance to alleviate overfitting and improve deep neural networks' generalization performance.
Liver Fibrosis and NAS scoring from CT images using self-supervised learning and texture encoding
The severity and treatment of NAFLD is determined by NAFLD Activity Scores (NAS)and liver fibrosis stage, which are usually obtained from liver biopsy.
DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images
Cardiac tagging magnetic resonance imaging (t-MRI) is the gold standard for regional myocardium deformation and cardiac strain estimation.
Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
A federatedGAN jointly trains a centralized generator and multiple private discriminators hosted at different sites.
CrossNorm and SelfNorm for Generalization under Distribution Shifts
Can we develop new normalization methods to improve generalization robustness under distribution shifts?
A Topological Filter for Learning with Label Noise
Noisy labels can impair the performance of deep neural networks.
Learning View-Disentangled Human Pose Representation by Contrastive Cross-View Mutual Information Maximization
To evaluate the power of the learned representations, in addition to the conventional fully-supervised action recognition settings, we introduce a novel task called single-shot cross-view action recognition.
Deep Subspace Clustering with Data Augmentation
In particular, we introduce a temporal ensembling component to the objective function of DSC algorithms to enable the DSC networks to maintain consistent subspaces for random transformations in the input data.
Error-Bounded Correction of Noisy Labels
To be robust against label noise, many successful methods rely on the noisy classifiers (i. e., models trained on the noisy training data) to determine whether a label is trustworthy.
Ranked #41 on
Image Classification
on Clothing1M
Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
In this paper, we propose a novel and effective regularization term for adversarial data augmentation.
Deep Learning based NAS Score and Fibrosis Stage Prediction from CT and Pathology Data
In this work, we propose a novel method to automatically predict NAS score and fibrosis stage from CT data that is non-invasive and inexpensive to obtain compared with liver biopsy.
Measure Anatomical Thickness from Cardiac MRI with Deep Neural Networks
We also analyze thickness patterns on different cardiac pathologies with a standard clinical model and the results demonstrate the potential clinical value of our method for thickness based cardiac disease diagnosis.
Enhanced MRI Reconstruction Network using Neural Architecture Search
The accurate reconstruction of under-sampled magnetic resonance imaging (MRI) data using modern deep learning technology, requires significant effort to design the necessary complex neural network architectures.
PC-U Net: Learning to Jointly Reconstruct and Segment the Cardiac Walls in 3D from CT Data
The 3D volumetric shape of the heart's left ventricle (LV) myocardium (MYO) wall provides important information for diagnosis of cardiac disease and invasive procedure navigation.
Oriented Object Detection in Aerial Images with Box Boundary-Aware Vectors
To address this issue, in this work we extend the horizontal keypoint-based object detector to the oriented object detection task.
Ranked #14 on
Oriented Object Detection
on DOTA 1.0
OnlineAugment: Online Data Augmentation with Less Domain Knowledge
First is that most if not all modern augmentation search methods are offline and learning policies are isolated from their usage.
Learn distributed GAN with Temporary Discriminators
Our proposed method tackles the challenge of training GAN in the federated learning manner: How to update the generator with a flow of temporary discriminators?
Unbiased Auxiliary Classifier GANs with MINE
To remedy this, Twin Auxiliary Classifier GAN (TAC-GAN) introduces a twin classifier to the min-max game.
Synthetic Learning: Learn From Distributed Asynchronized Discriminator GAN Without Sharing Medical Image Data
In this paper, we propose a data privacy-preserving and communication efficient distributed GAN learning framework named Distributed Asynchronized Discriminator GAN (AsynDGAN).
MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps
The backbone of MotionNet is a novel spatio-temporal pyramid network, which extracts deep spatial and temporal features in a hierarchical fashion.
Point Cloud Processing via Recurrent Set Encoding
The spatial layout of the beams is regular, and this allows the beam features to be further fed into an efficient 2D convolutional neural network (CNN) for hierarchical feature aggregation.
Robust Conditional GAN from Uncertainty-Aware Pairwise Comparisons
Conditional generative adversarial networks have shown exceptional generation performance over the past few years.
Label Cleaning with Likelihood Ratio Test
To collect large scale annotated data, it is inevitable to introduce label noise, i. e., incorrect class labels.
Unsupervised Domain Adaptation via Calibrating Uncertainties
Unsupervised domain adaptation (UDA) aims at inferring class labels for unlabeled target domain given a related labeled source dataset.
Effective 3D Humerus and Scapula Extraction using Low-contrast and High-shape-variability MR Data
For the initial shoulder preoperative diagnosis, it is essential to obtain a three-dimensional (3D) bone mask from medical images, e. g., magnetic resonance (MR).
Brain Segmentation from k-space with End-to-end Recurrent Attention Network
The task of medical image segmentation commonly involves an image reconstruction step to convert acquired raw data to images before any analysis.
MRI Reconstruction via Cascaded Channel-wise Attention Network
We consider an MRI reconstruction problem with input of k-space data at a very low undersampled rate.
Quantized Densely Connected U-Nets for Efficient Landmark Localization
Finally, to reduce the memory consumption and high precision operations both in training and testing, we further quantize weights, inputs, and gradients of our localization network to low bit-width numbers.
Ranked #19 on
Pose Estimation
on MPII Human Pose
Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
We consider the problem of image-to-video translation, where an input image is translated into an output video containing motions of a single object.
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
Random data augmentation is a critical technique to avoid overfitting in training deep neural network models.
Ranked #3 on
Pose Estimation
on Leeds Sports Poses
Self-Attention Generative Adversarial Networks
In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks.
Ranked #20 on
Conditional Image Generation
on ImageNet 128x128
(using extra training data)
Improving GANs Using Optimal Transport
We present Optimal Transport GAN (OT-GAN), a variant of generative adversarial nets minimizing a new metric measuring the distance between the generator distribution and the data distribution.
Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution
The results are also compared with state-of-the-art methods on HumanEva-I dataset, which demonstrates the superior performance of our approach.
Interactive Reinforcement Learning for Object Grounding via Self-Talking
In this paper, we introduce an interactive training method to improve the natural language conversation system for a visual grounding task.
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks
In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) aiming at generating high-resolution photo-realistic images.
Ranked #6 on
Text-to-Image Generation
on Oxford 102 Flowers
Automatic Liver Segmentation Using an Adversarial Image-to-Image Network
Automatic liver segmentation in 3D medical images is essential in many clinical applications, such as pathological diagnosis of hepatic diseases, surgical planning, and postoperative assessment.
Automatic Vertebra Labeling in Large-Scale 3D CT using Deep Image-to-Image Network with Message Passing and Sparsity Regularization
In this paper, we propose an automatic and fast algorithm to localize and label the vertebra centroids in 3D CT volumes.
Reconstruction-Based Disentanglement for Pose-invariant Face Recognition
Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject.
StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications.
Ranked #4 on
Text-to-Image Generation
on Oxford 102 Flowers
(Inception score metric)
SPDA-CNN: Unifying Semantic Part Detection and Abstraction for Fine-Grained Recognition
In this paper, we propose a new CNN architecture that integrates semantic part detection and abstraction (SPDA-CNN) for fine-grained classification.
Detection of Major ASL Sign Types in Continuous Signing For ASL Recognition
In American Sign Language (ASL) as well as other signed languages, different classes of signs (e. g., lexical signs, fingerspelled signs, and classifier constructions) have different internal structural properties.
Mode Estimation for High Dimensional Discrete Tree Graphical Models
Though the mode finding problem is generally intractable in high dimensions, this paper unveils that, if the distribution can be approximated well by a tree graphical model, mode characterization is significantly easier.
3D Face Tracking and Multi-Scale, Spatio-temporal Analysis of Linguistically Significant Facial Expressions and Head Positions in ASL
Essential grammatical information is conveyed in signed languages by clusters of events involving facial expressions and movements of the head and upper body.
A New Framework for Sign Language Recognition based on 3D Handshape Identification and Linguistic Modeling
Current approaches to sign recognition by computer generally have at least some of the following limitations: they rely on laboratory conditions for sign production, are limited to a small vocabulary, rely on 2D modeling (and therefore cannot deal with occlusions and off-plane rotations), and/or achieve limited success.
Handling Noise in Single Image Deblurring Using Directional Filters
Based on this observation, our method applies a series of directional filters at different orientations to the input image, and estimates an accurate Radon transform of the blur kernel from each filtered image.
Adaptive low rank and sparse decomposition of video using compressive sensing
We address the problem of reconstructing and analyzing surveillance videos using compressive sensing.
Recognition of Nonmanual Markers in American Sign Language (ASL) Using Non-Parametric Adaptive 2D-3D Face Tracking
This paper addresses the problem of automatically recognizing linguistically significant nonmanual expressions in American Sign Language from video.
