Search Results for author:
Found 78 papers, 41 papers with code
Recognition of Nonmanual Markers in American Sign Language (ASL) Using Non-Parametric Adaptive 2D-3D Face Tracking
This paper addresses the problem of automatically recognizing linguistically significant nonmanual expressions in American Sign Language from video.
Adaptive low rank and sparse decomposition of video using compressive sensing
We address the problem of reconstructing and analyzing surveillance videos using compressive sensing.
Handling Noise in Single Image Deblurring Using Directional Filters
Based on this observation, our method applies a series of directional filters at different orientations to the input image, and estimates an accurate Radon transform of the blur kernel from each filtered image.
A New Framework for Sign Language Recognition based on 3D Handshape Identification and Linguistic Modeling
Current approaches to sign recognition by computer generally have at least some of the following limitations: they rely on laboratory conditions for sign production, are limited to a small vocabulary, rely on 2D modeling (and therefore cannot deal with occlusions and off-plane rotations), and/or achieve limited success.
3D Face Tracking and Multi-Scale, Spatio-temporal Analysis of Linguistically Significant Facial Expressions and Head Positions in ASL
Essential grammatical information is conveyed in signed languages by clusters of events involving facial expressions and movements of the head and upper body.
Mode Estimation for High Dimensional Discrete Tree Graphical Models
Though the mode finding problem is generally intractable in high dimensions, this paper unveils that, if the distribution can be approximated well by a tree graphical model, mode characterization is significantly easier.
Detection of Major ASL Sign Types in Continuous Signing For ASL Recognition
In American Sign Language (ASL) as well as other signed languages, different classes of signs (e. g., lexical signs, fingerspelled signs, and classifier constructions) have different internal structural properties.
SPDA-CNN: Unifying Semantic Part Detection and Abstraction for Fine-Grained Recognition
In this paper, we propose a new CNN architecture that integrates semantic part detection and abstraction (SPDA-CNN) for fine-grained classification.
StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications.
Ranked #3 on
Text-to-Image Generation
on Oxford 102 Flowers
(Inception score metric)
Reconstruction-Based Disentanglement for Pose-invariant Face Recognition
Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject.
Automatic Vertebra Labeling in Large-Scale 3D CT using Deep Image-to-Image Network with Message Passing and Sparsity Regularization
In this paper, we propose an automatic and fast algorithm to localize and label the vertebra centroids in 3D CT volumes.
Automatic Liver Segmentation Using an Adversarial Image-to-Image Network
Automatic liver segmentation in 3D medical images is essential in many clinical applications, such as pathological diagnosis of hepatic diseases, surgical planning, and postoperative assessment.
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks
In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) aiming at generating high-resolution photo-realistic images.
Ranked #5 on
Text-to-Image Generation
on Oxford 102 Flowers
Interactive Reinforcement Learning for Object Grounding via Self-Talking
In this paper, we introduce an interactive training method to improve the natural language conversation system for a visual grounding task.
Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution
The results are also compared with state-of-the-art methods on HumanEva-I dataset, which demonstrates the superior performance of our approach.
Improving GANs Using Optimal Transport
We present Optimal Transport GAN (OT-GAN), a variant of generative adversarial nets minimizing a new metric measuring the distance between the generator distribution and the data distribution.
Self-Attention Generative Adversarial Networks
In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks.
Ranked #20 on
Conditional Image Generation
on ImageNet 128x128
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
Random data augmentation is a critical technique to avoid overfitting in training deep neural network models.
Ranked #3 on
Pose Estimation
on Leeds Sports Poses
Show Me a Story: Towards Coherent Neural Story Illustration
We propose an end-to-end network for the visual illustration of a sequence of sentences forming a story.
Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
We consider the problem of image-to-video translation, where an input image is translated into an output video containing motions of a single object.
Quantized Densely Connected U-Nets for Efficient Landmark Localization
Finally, to reduce the memory consumption and high precision operations both in training and testing, we further quantize weights, inputs, and gradients of our localization network to low bit-width numbers.
Ranked #19 on
Pose Estimation
on MPII Human Pose
MRI Reconstruction via Cascaded Channel-wise Attention Network
We consider an MRI reconstruction problem with input of k-space data at a very low undersampled rate.
Brain Segmentation from k-space with End-to-end Recurrent Attention Network
The task of medical image segmentation commonly involves an image reconstruction step to convert acquired raw data to images before any analysis.
Effective 3D Humerus and Scapula Extraction using Low-contrast and High-shape-variability MR Data
For the initial shoulder preoperative diagnosis, it is essential to obtain a three-dimensional (3D) bone mask from medical images, e. g., magnetic resonance (MR).
Unsupervised Domain Adaptation via Calibrating Uncertainties
Unsupervised domain adaptation (UDA) aims at inferring class labels for unlabeled target domain given a related labeled source dataset.
Label Cleaning with Likelihood Ratio Test
To collect large scale annotated data, it is inevitable to introduce label noise, i. e., incorrect class labels.
Robust Conditional GAN from Uncertainty-Aware Pairwise Comparisons
Conditional generative adversarial networks have shown exceptional generation performance over the past few years.
Point Cloud Processing via Recurrent Set Encoding
The spatial layout of the beams is regular, and this allows the beam features to be further fed into an efficient 2D convolutional neural network (CNN) for hierarchical feature aggregation.
MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps
The backbone of MotionNet is a novel spatio-temporal pyramid network, which extracts deep spatial and temporal features in a hierarchical fashion.
Synthetic Learning: Learn From Distributed Asynchronized Discriminator GAN Without Sharing Medical Image Data
In this paper, we propose a data privacy-preserving and communication efficient distributed GAN learning framework named Distributed Asynchronized Discriminator GAN (AsynDGAN).
Unbiased Auxiliary Classifier GANs with MINE
To remedy this, Twin Auxiliary Classifier GAN (TAC-GAN) introduces a twin classifier to the min-max game.
OnlineAugment: Online Data Augmentation with Less Domain Knowledge
First is that most if not all modern augmentation search methods are offline and learning policies are isolated from their usage.
Learn distributed GAN with Temporary Discriminators
Our proposed method tackles the challenge of training GAN in the federated learning manner: How to update the generator with a flow of temporary discriminators?
Oriented Object Detection in Aerial Images with Box Boundary-Aware Vectors
To address this issue, in this work we extend the horizontal keypoint-based object detector to the oriented object detection task.
Ranked #11 on
Oriented Object Detection
on DOTA 1.0
PC-U Net: Learning to Jointly Reconstruct and Segment the Cardiac Walls in 3D from CT Data
The 3D volumetric shape of the heart's left ventricle (LV) myocardium (MYO) wall provides important information for diagnosis of cardiac disease and invasive procedure navigation.
Enhanced MRI Reconstruction Network using Neural Architecture Search
The accurate reconstruction of under-sampled magnetic resonance imaging (MRI) data using modern deep learning technology, requires significant effort to design the necessary complex neural network architectures.
Measure Anatomical Thickness from Cardiac MRI with Deep Neural Networks
We also analyze thickness patterns on different cardiac pathologies with a standard clinical model and the results demonstrate the potential clinical value of our method for thickness based cardiac disease diagnosis.
Deep Learning based NAS Score and Fibrosis Stage Prediction from CT and Pathology Data
In this work, we propose a novel method to automatically predict NAS score and fibrosis stage from CT data that is non-invasive and inexpensive to obtain compared with liver biopsy.
Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
In this paper, we propose a novel and effective regularization term for adversarial data augmentation.
Error-Bounded Correction of Noisy Labels
To be robust against label noise, many successful methods rely on the noisy classifiers (i. e., models trained on the noisy training data) to determine whether a label is trustworthy.
Ranked #40 on
Image Classification
on Clothing1M
Deep Subspace Clustering with Data Augmentation
In particular, we introduce a temporal ensembling component to the objective function of DSC algorithms to enable the DSC networks to maintain consistent subspaces for random transformations in the input data.
Learning View-Disentangled Human Pose Representation by Contrastive Cross-View Mutual Information Maximization
To evaluate the power of the learned representations, in addition to the conventional fully-supervised action recognition settings, we introduce a novel task called single-shot cross-view action recognition.
A Topological Filter for Learning with Label Noise
Noisy labels can impair the performance of deep neural networks.
CrossNorm and SelfNorm for Generalization under Distribution Shifts
Can we develop new normalization methods to improve generalization robustness under distribution shifts?
Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
A federatedGAN jointly trains a centralized generator and multiple private discriminators hosted at different sites.
DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images
Cardiac tagging magnetic resonance imaging (t-MRI) is the gold standard for regional myocardium deformation and cardiac strain estimation.
Liver Fibrosis and NAS scoring from CT images using self-supervised learning and texture encoding
The severity and treatment of NAFLD is determined by NAFLD Activity Scores (NAS)and liver fibrosis stage, which are usually obtained from liver biopsy.
Enabling Data Diversity: Efficient Automatic Augmentation via Regularized Adversarial Training
Data augmentation has proved extremely useful by increasing training data variance to alleviate overfitting and improve deep neural networks' generalization performance.
UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation
In this study, we present UTNet, a simple yet powerful hybrid Transformer architecture that integrates self-attention into a convolutional neural network for enhancing medical image segmentation.
Dual Projection Generative Adversarial Networks for Conditional Image Generation
We then propose an improved cGAN model with Auxiliary Classification that directly aligns the fake and real conditionals $P(\text{class}|\text{image})$ by minimizing their $f$-divergence.
Global and Local Interpretation of black-box Machine Learning models to determine prognostic factors from early COVID-19 data
We explore one of the most recent techniques called symbolic metamodeling to find the mathematical expression of the machine learning models for COVID-19.
Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis from Lung CT Scans with Multi-Scale Guided Dense Attention
Computed Tomography (CT) plays an important role in monitoring radiation-induced Pulmonary Fibrosis (PF), where accurate segmentation of the PF lesions is highly desired for diagnosis and treatment follow-up.
AE-StyleGAN: Improved Training of Style-Based Auto-Encoders
StyleGANs have shown impressive results on data generation and manipulation in recent years, thanks to its disentangled style latent space.
ASL Video Corpora & Sign Bank: Resources Available through the American Sign Language Linguistic Research Project (ASLLRP)
These data have been used for many types of research in linguistics and in computer-based sign language recognition from video; examples of such research are provided in the latter part of this article.
Contrastive and Selective Hidden Embeddings for Medical Image Segmentation
Medical image segmentation has been widely recognized as a pivot procedure for clinical diagnosis, analysis, and treatment planning.
Modality Bank: Learn multi-modality images across data centers without sharing medical data
Multi-modality images have been widely used and provide comprehensive information for medical image analysis.
Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning
In addition, our model can extract visual information as suggested by the text prompt, e. g., "an object in image one is moving northeast", and generate corresponding videos.
Region Proposal Rectification Towards Robust Instance Segmentation of Biological Images
Top-down instance segmentation framework has shown its superiority in object detection compared to the bottom-up framework.
TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformers
Combining information from multi-view images is crucial to improve the performance and robustness of automated methods for disease diagnosis.
Global Matching with Overlapping Attention for Optical Flow Estimation
In this paper, inspired by the traditional matching-optimization methods where matching is introduced to handle large displacements before energy-based optimizations, we introduce a simple but effective global matching step before the direct regression and develop a learning-based matching-optimization framework, namely GMFlowNet.
Ranked #4 on
Optical Flow Estimation
on KITTI 2015
A Manifold View of Adversarial Risk
Assuming data lies in a manifold, we investigate two new types of adversarial risk, the normal adversarial risk due to perturbation along normal direction, and the in-manifold adversarial risk due to perturbation within the manifold.
DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Method
Joint 2D cardiac segmentation and 3D volume reconstruction are fundamental to building statistical cardiac anatomy models and understanding functional mechanisms from motion patterns.
A Dynamic Data Driven Approach for Explainable Scene Understanding
Suppose that an agent utilizing one or more sensors is placed in an unknown environment, and based on its sensory input, the agent needs to assign some label to the perceived scene.
Exploiting Unlabeled Data with Vision and Language Models for Object Detection
We propose a novel method that leverages the rich semantics available in recent vision and language models to localize and classify objects in unlabeled images, effectively generating pseudo labels for object detection.
Ranked #15 on
Open Vocabulary Object Detection
on MSCOCO
(using extra training data)
Automatic Tooth Segmentation from 3D Dental Model using Deep Learning: A Quantitative Analysis of what can be learnt from a Single 3D Dental Model
We conclude that the segmentation methods can learn a great deal of information from a single 3D tooth point cloud scan under suitable conditions e. g. data augmentation.
SINE: SINgle Image Editing with Text-to-Image Diffusion Models
We propose a novel model-based guidance built upon the classifier-free guidance so that the knowledge from the model trained on a single image can be distilled into the pre-trained diffusion model, enabling content creation even with one given image.
Diffusion Guided Domain Adaptation of Image Generators
Can a text-to-image diffusion model be used as a training objective for adapting a GAN generator to another domain?
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning
Diffusion models have achieved remarkable success in text-to-image generation, enabling the creation of high-quality images from text prompts or other modalities.
Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies
Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned.
OmniLabel: A Challenging Benchmark for Language-Based Object Detection
With more than 28K unique object descriptions on over 25K images, OmniLabel provides a challenging benchmark with diverse and complex object descriptions in a naturally open-vocabulary setting.
Learning Articulated Shape with Keypoint Pseudo-labels from Web Images
It is based on two key insights: (1) 2D keypoint estimation networks trained on as few as 50-150 images of a given object category generalize well and generate reliable pseudo-labels; (2) a data selection mechanism can automatically create a "curated" subset of the unlabeled web images that can be used for training -- we evaluate four data selection methods.
Improving Tuning-Free Real Image Editing with Proximal Guidance
Null-text inversion (NTI) optimizes null embeddings to align the reconstruction and inversion trajectories with larger CFG scales, enabling real image editing with cross-attention control.
On the Challenges and Perspectives of Foundation Models for Medical Image Analysis
This article discusses the opportunities, applications and future directions of large-scale pre-trained models, i. e., foundation models, for analyzing medical images.
Neural Deformable Models for 3D Bi-Ventricular Heart Shape Reconstruction and Modeling from 2D Sparse Cardiac Magnetic Resonance Imaging
We model the bi-ventricular shape using blended deformable superquadrics, which are parameterized by a set of geometric parameter functions and are capable of deforming globally and locally.
Improving Compositional Text-to-image Generation with Large Vision-Language Models
However, compositional text-to-image models frequently encounter difficulties in generating high-quality images that accurately align with input texts describing multiple objects, variable attributes, and intricate spatial relationships.
AVID: Any-Length Video Inpainting with Diffusion Model
Given a video, a masked region at its initial frame, and an editing prompt, it requires a model to do infilling at each frame following the editing guidance while keeping the out-of-mask region intact.
Score-Guided Diffusion for 3D Human Recovery
We present Score-Guided Human Mesh Recovery (ScoreHMR), an approach for solving inverse problems for 3D human pose and shape reconstruction.
