Search Results for author:
Found 357 papers, 221 papers with code
On a Theory of Nonparametric Pairwise Similarity for Clustering: Connecting Clustering to Classification
Modeling the underlying data distribution by nonparametric kernel density estimation, the generalization error bounds for both unsupervised nonparametric classifiers are the sum of nonparametric pairwise similarity terms between the data points for the purpose of clustering.
Decomposition-Based Domain Adaptation for Real-World Font Recognition
We present a domain adaption framework to address a domain mismatch between synthetic training and real-world testing data.
Learning Super-Resolution Jointly from External and Internal Examples
Single image super-resolution (SR) aims to estimate a high-resolution (HR) image from a lowresolution (LR) input.
Designing A Composite Dictionary Adaptively From Joint Examples
We study the complementary behaviors of external and internal examples in image restoration, and are motivated to formulate a composite dictionary design framework.
Real-World Font Recognition Using Deep Network and Domain Adaptation
We address a challenging fine-grain classification problem: recognizing a font style from an image of text.
Self-Tuned Deep Super Resolution
Deep learning has been successfully applied to image super resolution (SR).
DeepFont: Identify Your Font from An Image
As font is one of the core design concepts, automatic font identification and similar font suggestion from an image or photo has been on the wish list of many designers.
Ranked #1 on
Font Recognition
on VFR-Wild
Learning Deep $\ell_0$ Encoders
We study the $\ell_0$ sparse approximation problem with the tool of deep learning, by proposing Deep $\ell_0$ Encoders.
Learning A Task-Specific Deep Architecture For Clustering
In this paper, we propose to emulate the sparse coding-based clustering pipeline in the context of deep learning, leading to a carefully crafted deep model benefiting from both.
$\mathbf{D^3}$: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain ($\mathbf{D^3}$) based fast restoration model to remove artifacts of JPEG compressed images.
Studying Very Low Resolution Recognition Using Deep Networks
Visual recognition research often assumes a sufficient resolution of the region of interest (ROI).
Brain-Inspired Deep Networks for Image Aesthetics Assessment
Image aesthetics assessment has been challenging due to its subjective nature.
Learning A Deep $\ell_\infty$ Encoder for Hashing
We investigate the $\ell_\infty$-constrained representation which demonstrates robustness to quantization errors, utilizing the tool of deep learning.
D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain (D3) based fast restoration model to remove artifacts of JPEG compressed images.
UnitBox: An Advanced Object Detection Network
In present object detection systems, the deep convolutional neural networks (CNNs) are utilized to predict bounding boxes of object candidates, and have gained performance advantages over the traditional region proposal methods.
Stacked Approximated Regression Machine: A Simple Deep Learning Approach
With the agreement of my coauthors, I Zhangyang Wang would like to withdraw the manuscript "Stacked Approximated Regression Machine: A Simple Deep Learning Approach".
Deep Double Sparsity Encoder: Learning to Sparsify Not Only Features But Also Parameters
This paper emphasizes the significance to jointly exploit the problem structure and the parameter structure, in the context of deep modeling.
When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach
Conventionally, image denoising and high-level vision tasks are handled separately in computer vision.
An All-in-One Network for Dehazing and Beyond
This paper proposes an image dehazing model built with a convolutional neural network (CNN), called All-in-One Dehazing Network (AOD-Net).
Robust Emotion Recognition from Low Quality and Low Bit Rate Video: A Deep Learning Approach
Emotion recognition from facial expressions is tremendously useful, especially when coupled with smart devices and wireless multimedia applications.
End-to-End United Video Dehazing and Detection
Furthermore, we build an End-to-End United Video Dehazing and Detection Network(EVDD-Net), which concatenates and jointly trains EVD-Net with a video object detection model.
AOD-Net: All-In-One Dehazing Network
This paper proposes an image dehazing model built with a convolutional neural network (CNN), called All-in-One Dehazing Network (AOD-Net).
Ranked #20 on
Image Dehazing
on SOTS Outdoor
Robust Video Super-Resolution With Learned Temporal Dynamics
Second, we reduce the complexity of motion between neighboring frames using a spatial alignment network that is much more robust and efficient than competing alignment methods and can be jointly trained with the temporal adaptive network in an end-to-end manner.
Predicting Depression Severity by Multi-Modal Feature Engineering and Fusion
We present our preliminary work to determine if patient's vocal acoustic, linguistic, and facial patterns could predict clinical ratings of depression severity, namely Patient Health Questionnaire depression scale (PHQ-8).
Benchmarking Single Image Dehazing and Beyond
We present a comprehensive study and evaluation of existing single image dehazing algorithms, using a new large-scale benchmark consisting of both synthetic and real-world hazy images, called REalistic Single Image DEhazing (RESIDE).
Enhance Visual Recognition under Adverse Conditions via Deep Networks
Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage.
Subspace Network: Deep Multi-Task Censored Regression for Modeling Neurodegenerative Diseases
Over the past decade a wide spectrum of machine learning models have been developed to model the neurodegenerative diseases, associating biomarkers, especially non-intrusive neuroimaging markers, with key clinical scores measuring the cognitive status of patients.
Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding
We propose an interpretable deep structure namely Frank-Wolfe Network (F-W Net), whose architecture is inspired by unrolling and truncating the Frank-Wolfe algorithm for solving an $L_p$-norm constrained problem with $p\geq 1$.
Learning Simple Thresholded Features with Sparse Support Recovery
The thresholded feature has recently emerged as an extremely efficient, yet rough empirical approximation, of the time-consuming sparse coding inference process.
DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networks
Motivation: Drug discovery demands rapid quantification of compound-protein interaction (CPI).
Ranked #2 on
Drug Discovery
on BindingDB IC50
Deep $k$-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions
The current trend of pushing CNNs deeper with convolutions has created a pressing demand to achieve higher compression gains on CNNs where convolutions dominate the computation and parameter amount (e. g., GoogLeNet, ResNet and Wide ResNet).
Improved Techniques for Learning to Dehaze and Beyond: A Collective Study
Here we explore two related but important tasks based on the recently released REalistic Single Image DEhazing (RESIDE) benchmark dataset: (i) single image dehazing as a low-level image restoration problem; and (ii) high-level visual understanding (e. g., object detection) of hazy images.
Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions
The current trend of pushing CNNs deeper with convolutions has created a pressing demand to achieve higher compression gains on CNNs where convolutions dominate the computation and parameter amount (e. g., GoogLeNet, ResNet and Wide ResNet).
Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study
This paper aims to improve privacy-preserving visual recognition, an increasingly demanded feature in smart camera applications, by formulating a unique adversarial training framework.
U-Finger: Multi-Scale Dilated Convolutional Network for Fingerprint Image Denoising and Inpainting
This paper studies the challenging problem of fingerprint image denoising and inpainting.
Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds
In this work, we study unfolded ISTA (Iterative Shrinkage Thresholding Algorithm) for sparse signal recovery.
DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification
Given insufficient data, while many techniques have been developed to help combat overfitting, the challenge remains if one tries to train deep networks, especially in the ill-posed extremely low data regimes: only a small set of labeled data are available, and nothing -- including unlabeled data -- else.
Connecting Image Denoising and High-Level Vision Tasks via Deep Learning
Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via back-propagation.
EnergyNet: Energy-Efficient Dynamic Inference
The prohibitive energy cost of running high-performance Convolutional Neural Networks (CNNs) has been limiting their deployment on resource-constrained platforms including mobile and wearable devices.
Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
This paper seeks to answer the question: as the (near-) orthogonality of weights is found to be a favorable property for training deep convolutional neural networks, how can we enforce it in more effective and easy-to-use ways?
Can We Gain More from Orthogonality Regularizations in Training Deep Networks?
This paper seeks to answer the question: as the (near-) orthogonality of weights is found to be a favorable property for training deep convolutional neural networks, how can we enforce it in more effective and easy-to-use ways?
Adaptive Activity Monitoring with Uncertainty Quantification in Switching Gaussian Process Models
This power-efficient sensing scheme can be achieved by deciding which group of sensors to use at a given time, requiring an accurate characterization of the trade-off between sensor energy usage and the uncertainty in ignoring certain sensor signals while monitoring.
Bridging the Gap Between Computational Photography and Visual Recognition
From the observed results, it is evident that we are in the early days of building a bridge between computational photography and visual recognition, leaving many opportunities for innovation in this area.
Model Compression with Adversarial Robustness: A Unified Optimization Framework
Deep model compression has been extensively studied, and state-of-the-art methods can now achieve high compression ratios with minimal accuracy loss.
Convolutional Sparse Coding for Compressed Sensing CT Reconstruction
Over the past few years, dictionary learning (DL)-based methods have been successfully used in various image reconstruction problems.
Single Image Deraining: A Comprehensive Benchmark Analysis
We present a comprehensive study and evaluation of existing single image deraining algorithms, using a new large-scale benchmark consisting of both synthetic and real-world rainy images. This dataset highlights diverse data sources and image contents, and is divided into three subsets (rain streak, rain drop, rain and mist), each serving different training or evaluation purposes.
UG$^{2+}$ Track 2: A Collective Benchmark Effort for Evaluating and Advancing Image Understanding in Poor Visibility Environments
The UG$^{2+}$ challenge in IEEE CVPR 2019 aims to evoke a comprehensive discussion and exploration about how low-level vision techniques can benefit the high-level automatic visual recognition in various scenarios.
ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA
In this work, we propose Analytic LISTA (ALISTA), where the weight matrix in LISTA is computed as the solution to a data-free optimization problem, leaving only the stepsize and threshold parameters to data-driven learning.
Controllable Artistic Text Style Transfer via Shape-Matching GAN
In this paper, we present the first text style transfer network that allows for real-time control of the crucial stylistic degree of the glyph through an adjustable parameter.
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
Plug-and-play (PnP) is a non-convex framework that integrates modern denoising priors, such as BM3D or deep learning-based denoisers, into ADMM or other proximal algorithms.
Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images
In either way, the loss of local fine details or global contextual information results in limited segmentation accuracy.
Ranked #4 on
Land Cover Classification
on DeepGlobe
Predicting Model Failure using Saliency Maps in Autonomous Driving Systems
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations.
Segmentation-Aware Image Denoising without Knowing True Segmentation
Several recent works discussed application-driven image restoration neural networks, which are capable of not only removing noise in images but also preserving their semantic-aware details, making them suitable for various high-level computer vision tasks as the pre-processing step.
All-In-One Underwater Image Enhancement using Domain-Adversarial Learning
We train our model on a dataset consisting images of 10 Jerlov water types.
ArcticNet: A Deep Learning Solution to Classify Arctic Wetlands
Arctic environments are rapidly changing under the warming climate.
Privacy-Preserving Deep Action Recognition: An Adversarial Learning Framework and A New Dataset
We first discuss an innovative heuristic of cross-dataset training and evaluation, enabling the use of multiple single-task datasets (one with target task labels and the other with privacy labels) in our problem.
EnlightenGAN: Deep Light Enhancement without Paired Supervision
Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data?
Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference
State-of-the-art convolutional neural networks (CNNs) yield record-breaking predictive performance, yet at the cost of high-energy-consumption inference, that prohibits their widely deployments in resource-constrained Internet of Things (IoT) applications.
Report on UG^2+ Challenge Track 1: Assessing Algorithms to Improve Video Object Detection and Classification from Unconstrained Mobility Platforms
How can we effectively engineer a computer vision system that is able to interpret videos from unconstrained mobility platforms like UAVs?
ABD-Net: Attentive but Diverse Person Re-Identification
Attention mechanism has been shown to be effective for person re-identification (Re-ID).
Ranked #16 on
Person Re-Identification
on Market-1501-C
DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
We present a new end-to-end generative adversarial network (GAN) for single image motion deblurring, named DeblurGAN-v2, which considerably boosts state-of-the-art deblurring efficiency, quality, and flexibility.
Ranked #3 on
Blind Face Restoration
on CelebA-Test
AutoGAN: Neural Architecture Search for Generative Adversarial Networks
Neural architecture search (NAS) has witnessed prevailing success in image classification and (very recently) segmentation tasks.
Ranked #16 on
Image Generation
on STL-10
Delving into Robust Object Detection from Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach
Object detection from images captured by Unmanned Aerial Vehicles (UAVs) is becoming increasingly useful.
Universal Safeguarded Learned Convex Optimization with Guaranteed Convergence
Inferences by each network form solution estimates, and networks are trained to optimize these estimates for a particular distribution of data.
Neural Networks for Principal Component Analysis: A New Loss Function Provably Yields Ordered Exact Eigenvectors
In this paper, we propose a new loss function for performing principal component analysis (PCA) using linear autoencoders (LAEs).
Is There Mode Collapse? A Case Study on Face Generation and Its Black-box Calibration
Generative adversarial networks (GANs) nowadays are capable of producing im-ages of incredible realism.
Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networks
In this paper, we discover for the first time that the winning tickets can be identified at the very early training stage, which we term as early-bird (EB) tickets, via low-cost training schemes (e. g., early stopping and low-precision training) at large learning rates.
E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings
Extensive simulations and ablation studies, with real energy measurements from an FPGA board, confirm the superiority of our proposed strategies and demonstrate remarkable energy savings for training.
Learning to Optimize in Swarms
Learning to optimize has emerged as a powerful framework for various optimization and machine learning tasks.
Calibrated Domain-Invariant Learning for Highly Generalizable Large Scale Re-Identification
Many real-world applications, such as city-scale traffic monitoring and control, requires large-scale re-identification.
DAVID: Dual-Attentional Video Deblurring
To train and evaluate on more diverse blur severity levels, we propose a Challenging DVD dataset generated from the raw DVD video set by pooling frames with different temporal windows.
In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation
This work addresses the above two shortcomings of triplet loss, extending its effectiveness to large-scale ReID datasets with potentially noisy labels.
Practical Solutions for Machine Learning Safety in Autonomous Vehicles
Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning.
FasterSeg: Searching for Faster Real-time Semantic Segmentation
We present FasterSeg, an automatically designed semantic segmentation network with not only state-of-the-art performance but also faster speed than current methods.
Ranked #1 on
Semantic Segmentation
on BDD
Neural Architecture Search
Real-Time Semantic Segmentation
+1
Explainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts
DeepRelations shows superior interpretability to the state-of-the-art: without compromising affinity prediction, it boosts the AUPRC of contact prediction 9. 5, 16. 9, 19. 3 and 5. 7-fold for the test, compound-unique, protein-unique, and both-unique sets, respectively.
Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference
The core idea of DFS is to hypothesize layer-wise quantization (to different bitwidths) as intermediate "soft" choices to be made between fully utilizing and skipping a layer.
Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches
We present a sketch refinement strategy, as inspired by the coarse-to-fine drawing process of the artists, which we show can help our model well adapt to casual and varied sketches without the need for real sketch training data.
VGAI: End-to-End Learning of Vision-Based Decentralized Controllers for Robot Swarms
More specifically, we consider that each robot has access to a visual perception of the immediate surroundings, and communication capabilities to transmit and receive messages from other neighboring robots.
Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inference
Deep networks were recently suggested to face the odds between accuracy (on clean natural images) and robustness (on adversarially perturbed images) (Tsipras et al., 2019).
I Am Going MAD: Maximum Discrepancy Competition for Comparing Classifiers Adaptively
On the other hand, the trained classifiers have traditionally been evaluated on small and fixed sets of test images, which are deemed to be extremely sparsely distributed in the space of all natural images.
Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery
Activity recognition in wearable computing faces two key challenges: i) activity characteristics may be context-dependent and change under different contexts or situations; ii) unknown contexts and activities may occur from time to time, requiring flexibility and adaptability of the algorithm.
Safeguarded Learned Convex Optimization
Our numerical examples show convergence of Safe-L2O algorithms, even when the provided data is not from the distribution of training data.
Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
We conduct extensive experiments to demonstrate that the proposed framework achieves large performance margins (eg, 3. 83% on robust accuracy and 1. 3% on standard accuracy, on the CIFAR-10 dataset), compared with the conventional end-to-end adversarial training baseline.
L^2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks
Graph convolution networks (GCN) are increasingly popular in many applications, yet remain notoriously hard to train over large graph datasets.
Drawing Early-Bird Tickets: Toward More Efficient Training of Deep Networks
Finally, we leverage the existence of EB tickets and the proposed mask distance to develop efficient training methods, which are achieved by first identifying EB tickets via low-cost schemes, and then continuing to train merely the EB tickets towards the target accuracy.
SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation
We present SmartExchange, an algorithm-hardware co-design framework to trade higher-cost memory storage/access for lower-cost computation, for energy-efficient inference of deep neural networks (DNNs).
AutoSpeech: Neural Architecture Search for Speaker Recognition
Speaker recognition systems based on Convolutional Neural Networks (CNNs) are often built with off-the-shelf backbones such as VGG-Net or ResNet.
Ranked #6 on
Speaker Identification
on VoxCeleb1
(using extra training data)
Focus Longer to See Better:Recursively Refined Attention for Fine-Grained Image Classification
In this paper, we tried to focus on these marginal differences to extract more representative features.
NADS: Neural Architecture Distribution Search for Uncertainty Awareness
Machine learning (ML) systems often encounter Out-of-Distribution (OoD) errors when dealing with testing data coming from a distribution different from training data.
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks
Inspired by the recent success of AutoML in deep compression, we introduce AutoML to GAN compression and develop an AutoGAN-Distiller (AGD) framework.
When Does Self-Supervision Help Graph Convolutional Networks?
We first elaborate three mechanisms to incorporate self-supervision into GCNs, analyze the limitations of pretraining & finetuning and self-training, and proceed to focus on multi-task learning.
Self-PU: Self Boosted and Calibrated Positive-Unlabeled Training
Many real-world applications have to tackle the Positive-Unlabeled (PU) learning problem, i. e., learning binary classifiers from a large amount of unlabeled data and a few labeled positive examples.
Can 3D Adversarial Logos Cloak Humans?
Contrary to the traditional adversarial patch, this new form of attack is mapped into the 3D object world and back-propagates to the 2D image domain through differentiable rendering.
Automated Synthetic-to-Real Generalization
Models trained on synthetic images often face degraded generalization to real data.
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
For a range of downstream tasks, we indeed find matching subnetworks at 40% to 90% sparsity.
AutoPose: Searching Multi-Scale Branch Aggregation for Pose Estimation
Such simplification limits the fusion of information at different scales and fails to maintain high-resolution representations.
GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework
Generative adversarial networks (GANs) have gained increasing popularity in various computer vision applications, and recently start to be deployed to resource-constrained mobile devices.
MM-Hand: 3D-Aware Multi-Modal Guided Hand Generative Network for 3D Hand Pose Synthesis
Estimating the 3D hand pose from a monocular RGB image is important but challenging.
PCAL: A Privacy-preserving Intelligent Credit Risk Modeling Framework Based on Adversarial Learning
While machine learning is increasingly used in this field, the resulting large-scale collection of user private information has reinvigorated the privacy debate, considering dozens of data breach incidents every year caused by unauthorized hackers, and (potentially even more) information misuse/abuse by authorized parties.
Training Stronger Baselines for Learning to Optimize
Learning to optimize (L2O) has gained increasing attention since classical optimizers require laborious problem-specific design and hyperparameter tuning.
Graph Contrastive Learning with Augmentations
In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data.
Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free
The trained model could be adjusted among different standard and robust accuracies "for free" at testing time.
ShiftAddNet: A Hardware-Inspired Deep Network
Multiplication (e. g., convolution) is arguably a cornerstone of modern deep neural networks (DNNs).
Robust Pre-Training by Adversarial Contrastive Learning
Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness In this work, we improve robustness-aware self-supervised pre-training by learning representations that are consistent under both data augmentations and adversarial perturbations.
What Does CNN Shift Invariance Look Like? A Visualization Study
We present the results of three experiments comparing representations of millions of images with exhaustively shifted objects, examining both local invariance (within a few pixels) and global invariance (across the image frame).
Uncertainty-Aware Physically-Guided Proxy Tasks for Unseen Domain Face Anti-spoofing
Face anti-spoofing (FAS) seeks to discriminate genuine faces from fake ones arising from any type of spoofing attack.
MATE: Plugging in Model Awareness to Task Embedding for Meta Learning
Meta-learning improves generalization of machine learning models when faced with previously unseen tasks by leveraging experiences from different, yet related prior tasks.
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
We extend the scope of LTH and question whether matching subnetworks still exist in pre-trained computer vision models, that enjoy the same downstream transfer performance.
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
Recent breakthroughs in deep neural networks (DNNs) have fueled a tremendous demand for intelligent edge devices featuring on-site learning, while the practical realization of such systems remains a challenge due to the limited resources available at the edge and the required massive training costs for state-of-the-art (SOTA) DNNs.
Growing Deep Forests Efficiently with Soft Routing and Learned Connectivity
For example, our model with only 1 layer of 15 trees can perform comparably with the model in [3] with 2 layers of 2000 trees each.
EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
Heavily overparameterized language models such as BERT, XLNet and T5 have achieved impressive success in many NLP tasks.
Efficiently Troubleshooting Image Segmentation Models with Human-In-The-Loop
Image segmentation lays the foundation for many high-stakes vision applications such as autonomous driving and medical image analysis.
Weak NAS Predictor Is All You Need
Rather than expecting a single strong predictor to model the whole space, we seek a progressive line of weak predictors that can connect a path to the best architecture, thus greatly simplifying the learning task of each predictor.
Dynamic Feature Selection for Efficient and Interpretable Human Activity Recognition
In many machine learning tasks, input features with varying degrees of predictive capability are usually acquired at some cost.
Robust Overfitting may be mitigated by properly learned smoothening
A recent study (Rice et al., 2020) revealed overfitting to be a dominant phenomenon in adversarially robust training of deep networks, and that appropriate early-stopping of adversarial training (AT) could match the performance gains of most recent algorithmic improvements.
Bayesian Learning to Optimize: Quantifying the Optimizer Uncertainty
Optimizing an objective function with uncertainty awareness is well-known to improve the accuracy and confidence of optimization solutions.
UMEC: Unified model and embedding compression for efficient recommendation systems
The recommendation system (RS) plays an important role in the content recommendation and retrieval scenarios.
On Dynamic Noise Influence in Differential Private Learning
In this paper, we provide comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions.
ALFA: Adversarial Feature Augmentation for Enhanced Image Recognition
Adversarial training is an effective method to combat adversarial attacks in order to create robust neural networks.
Learning A Minimax Optimizer: A Pilot Study
We first present Twin L2O, the first dedicated minimax L2O framework consisting of two LSTMs for updating min and max variables, respectively.
Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning
In view of those, we introduce two pruning options, e. g., top-down and bottom-up, for finding lifelong tickets.
SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training
Results show that: 1) applied to inference, SD achieves up to 2. 44x energy efficiency as evaluated via real hardware implementations; 2) applied to training, SD leads to 10. 56x and 4. 48x reduction in the storage and training energy, with negligible accuracy loss compared to state-of-the-art training baselines.
Spending Your Winning Lottery Better After Drawing It
In this paper, we demonstrate that it is unnecessary for spare retraining to strictly inherit those properties from the dense network.
Dynamic Privacy Budget Allocation Improves Data Efficiency of Differentially Private Gradient Descent
In this paper, we provide comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions.
A Unified Lottery Ticket Hypothesis for Graph Neural Networks
With graphs rapidly growing in size and deeper graph neural networks (GNNs) emerging, the training and inference of GNNs become increasingly expensive.
TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
Our vanilla GAN architecture, dubbed TransGAN, consists of a memory-friendly transformer-based generator that progressively increases feature resolution, and correspondingly a multi-scale discriminator to capture simultaneously semantic contexts and low-level textures.
Ranked #8 on
Image Generation
on STL-10
Stronger NAS with Weaker Predictors
We propose a paradigm shift from fitting the whole architecture space using one strong predictor, to progressively fitting a search path towards the high-performance sub-space through a set of weaker predictors.
Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity
We present Sandwich Batch Normalization (SaBN), a frustratingly easy improvement of Batch Normalization (BN) with only a few lines of code changes.
Ranked #20 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-100
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective
Can we select the best neural architectures without involving any training and eliminate a drastic portion of the search cost?
Exposing Semantic Segmentation Failures via Maximum Discrepancy Competition
A natural question then arises: Does the superior performance on the closed (and frequently re-used) test sets transfer to the open visual world with unconstrained variations?
Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
Training generative adversarial networks (GANs) with limited real image data generally results in deteriorated performance and collapsed models.
Adversarial Feature Augmentation and Normalization for Visual Recognition
Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models.
UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution
In this work, we propose UltraSR, a simple yet effective new network design based on implicit image functions in which we deeply integrated spatial coordinates and periodic encoding with the implicit neural representation.
Learning to Optimize: A Primer and A Benchmark
It automates the design of an optimization method based on its performance on a set of training problems.
The Elastic Lottery Ticket Hypothesis
Based on these results, we articulate the Elastic Lottery Ticket Hypothesis (E-LTH): by mindfully replicating (or dropping) and re-ordering layers for one network, its corresponding winning ticket could be stretched (or squeezed) into a subnetwork for another deeper (or shallower) network from the same family, whose performance is nearly the same competitive as the latter's winning ticket directly found by IMP.
Contrastive Syn-to-Real Generalization
Training on synthetic data can be beneficial for label or data-scarce scenarios.
Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data
Electronic Health Record (EHR) data has been of tremendous utility in Artificial Intelligence (AI) for healthcare such as predicting future clinical events.
A Design Space Study for LISTA and Beyond
Unrolling is believed to incorporate the model-based prior with the learning capacity of deep learning.
Knowledge-Augmented Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop
The key knob of our framework is a unique positive sampling approach tailored for the medical images, by seamlessly integrating radiomic features as a knowledge augmentation.
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
However, the BN layer is costly to calculate and is typically implemented with non-binary parameters, leaving a hurdle for the efficient implementation of BNN training.
Ranked #167 on
Image Classification
on CIFAR-10
InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks
The promise of Deep Neural Network (DNN) powered Internet of Thing (IoT) devices has motivated a tremendous demand for automated solutions to enable fast development and deployment of efficient (1) DNNs equipped with instantaneous accuracy-efficiency trade-off capability to accommodate the time-varying resources at IoT devices and (2) dataflows to optimize DNNs' execution efficiency on different devices.
Learning Transferable 3D Adversarial Cloaks for Deep Trained Detectors
This paper presents a novel patch-based adversarial attack pipeline that trains adversarial patches on 3D human meshes.
SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization
We demonstrate that plugging SAFIN into the base network of another state-of-the-art method results in enhanced stylization.
Troubleshooting Blind Image Quality Models in the Wild
Recently, the group maximum differentiation competition (gMAD) has been used to improve blind image quality assessment (BIQA) models, with the help of full-reference metrics.
Undistillable: Making A Nasty Teacher That CANNOT teach students
Knowledge Distillation (KD) is a widely used technique to transfer knowledge from pre-trained teacher models to (usually more lightweight) student models.
Improving Contrastive Learning on Imbalanced Data via Open-World Sampling
Contrastive learning approaches have achieved great success in learning visual representations with few labels of the target classes.
Efficient Lottery Ticket Finding: Less Data is More
We observe that a high-quality winning ticket can be found with training and pruning the dense network on the very compact PrAC set, which can substantially save training iterations for the ticket finding process.
Self-Damaging Contrastive Learning
Hence, the key innovation in SDCLR is to create a dynamic self-competitor model to contrast with the target model, which is a pruned version of the latter.
Chasing Sparsity in Vision Transformers: An End-to-End Exploration
For example, our sparsified DeiT-Small at (5%, 50%) sparsity for (data, architecture), improves 0. 28% top-1 accuracy, and meanwhile enjoys 49. 32% FLOPs and 4. 40% running time savings.
Ranked #20 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
Taxonomy of Machine Learning Safety: A Survey and Primer
The open-world deployment of Machine Learning (ML) algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities such as interpretability, verifiability, and performance limitations.
Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm
Compared to state-of-the-art methods, our homotopy attack leads to significantly fewer perturbations, e. g., reducing 42. 91% on CIFAR-10 and 75. 03% on ImageNet (average case, targeted attack), at similar maximal perturbation magnitudes, when still achieving 100% attack success rates.
Graph Contrastive Learning Automated
Unfortunately, unlike its counterpart on image data, the effectiveness of GraphCL hinges on ad-hoc data augmentations, which have to be manually picked per dataset, by either rules of thumb or trial-and-errors, owing to the diverse nature of graph data.
Federated Robustness Propagation: Sharing Robustness in Heterogeneous Federated Learning
In this paper, we study a novel FL strategy: propagating adversarial robustness from rich-resource users that can afford AT, to those with poor resources that cannot afford it, during federated learning.
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and before-training pruning (pruning at initialization).
Ranked #3 on
Sparse Learning
on ImageNet
IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers
The self-attention-based model, transformer, is recently becoming the leading backbone in the field of computer vision.
Ranked #29 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks
Our framework is implemented by a cascade of a convolutional and a graph neural network (CNN / GNN), addressing agent-level visual perception and feature learning, as well as swarm-level communication, local information aggregation and agent action inference, respectively.
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
Our framework, FreeTickets, is defined as the ensemble of these relatively cheap sparse subnetworks.
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Based on our analysis, we summarize a guideline for parameter settings in regards of specific architecture characteristics, which we hope to catalyze the research progress on the topic of lottery ticket hypothesis.
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
Semantic segmentation for scene understanding is nowadays widely demanded, raising significant challenges for the algorithm efficiency, especially its applications on resource-limited platforms.
Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study
Existing diversity tests of samples from GANs are usually conducted qualitatively on a small scale, and/or depends on the access to original training data as well as the trained model parameters.
CERL: A Unified Optimization Framework for Light Enhancement with Realistic Noise
We present \underline{C}oordinated \underline{E}nhancement for \underline{R}eal-world \underline{L}ow-light Noisy Images (CERL), that seamlessly integrates light enhancement and noise suppression parts into a unified and physics-grounded optimization framework.
Federated Adversarial Debiasing for Fair and Transferable Representations
While adversarial learning is commonly used in centralized learning for mitigating bias, there are significant barriers when extending it to the federated framework.
SSH: A Self-Supervised Framework for Image Harmonization
Image harmonization aims to improve the quality of image compositing by matching the "appearance" (\eg, color tone, brightness and contrast) between foreground and background images.
Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study
In view of those, we present the first fair and reproducible benchmark dedicated to assessing the "tricks" of training deep GNNs.
Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics
This work targets designing a principled and unified training-free framework for Neural Architecture Search (NAS), with high performance, low cost, and in-depth interpretation.
Font Completion and Manipulation by Cycling Between Multi-Modality Representations
The novel graph constructor maps a glyph's latent code to its graph representation that matches expert knowledge, which is trained to help the translation task.
Searching for Two-Stream Models in Multivariate Space for Video Recognition
We design a multivariate search space, including 6 search variables to capture a wide variety of choices in designing two-stream models.
GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization
During the training process, the polarization effect will drive a subset of gates to smoothly decrease to exact zero, while other gates gradually stay away from zero by a large margin.
Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs
We propose Skeleton-Graph, a deep spatio-temporal graph CNN model that predicts the future 3D skeleton poses in a single pass from the 2D ones.
Ranked #1 on
Trajectory Prediction
on PROX
Generalizable Learning to Optimize into Wide Valleys
Learning to optimize (L2O) has gained increasing popularity in various optimization tasks, since classical optimizers usually require laborious, problem-specific design and hyperparameter tuning.
CheXT: Knowledge-Guided Cross-Attention Transformer for Abnormality Classification and Localization in Chest X-rays
During training, the image branch leverages its learned attention to estimate pathology localization, which is then utilized to extract radiomic features from images in the radiomics branch.
Bayesian Modeling and Uncertainty Quantification for Learning to Optimize: What, Why, and How
Optimizing an objective function with uncertainty awareness is well-known to improve the accuracy and confidence of optimization solutions.
Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently
In this work, we define an extended class of subnetworks in randomly initialized NNs called disguised subnetworks, which are not only "hidden" in the random networks but also "disguised" -- hence can only be "unmasked" with certain transformations on weights.
Lottery Image Prior
Drawing inspirations from the recently prosperous research on lottery ticket hypothesis (LTH), we conjecture and study a novel “lottery image prior” (LIP), stated as: given an (untrained or trained) DNN-based image prior, it will have a sparse subnetwork that can be training in isolation, to match the original DNN’s performance when being applied as a prior to various image inverse problems.
Equalized Robustness: Towards Sustainable Fairness Under Distributional Shifts
In this paper, we first propose a new fairness goal, termed Equalized Robustness (ER), to impose fair model robustness against unseen distribution shifts across majority and minority groups.
Scaling the Depth of Vision Transformers via the Fourier Domain Analysis
The first technique, termed AttnScale, decomposes a self-attention block into low-pass and high-pass components, then rescales and combines these two filters to produce an all-pass self-attention matrix.
Reasoning With Hierarchical Symbols: Reclaiming Symbolic Policies For Visual Reinforcement Learning
Deep vision models are nowadays widely integrated into visual reinforcement learning (RL) to parameterize the policy networks.
Audio Lottery: Speech Recognition Made Ultra-Lightweight, Noise-Robust, and Transferable
In this paper, we investigate the tantalizing possibility of using lottery ticket hypothesis to discover lightweight speech recognition models, that are (1) robust to various noise existing in speech; (2) transferable to fit the open-world personalization; and 3) compatible with structured sparsity.
Universality of Deep Neural Network Lottery Tickets: A Renormalization Group Perspective
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possibly even across different architectures.
AutoCoG: A Unified Data-Modal Co-Search Framework for Graph Neural Networks
Despite the preliminary success, we argue that for GNNs, NAS has to be customized further, due to the topological complicacy of GNN input data (graph) as well as the notorious training instability.
Stingy Teacher: Sparse Logits Suffice to Fail Knowledge Distillation
However, it is unclear why the distorted distribution of the logits is catastrophic to the student model.
Lottery Tickets can have Structural Sparsity
The lottery ticket hypothesis (LTH) has shown that dense models contain highly sparse subnetworks (i. e., $\textit{winning tickets}$) that can be trained in isolation to match full accuracy.
Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining
Conventional methods often require (iterative) pruning followed by re-training, which not only incurs large overhead beyond the original DNN training but also can be sensitive to retraining hyperparameters.
Universality of Winning Tickets: A Renormalization Group Perspective
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possibly even across different architectures.
Towards Lifelong Learning of Multilingual Text-To-Speech Synthesis
This work presents a lifelong learning approach to train a multilingual Text-To-Speech (TTS) system, where each language was seen as an individual task and was learned sequentially and continually.
AugMax: Adversarial Composition of Random Augmentations for Robust Training
Diversity and hardness are two complementary dimensions of data augmentation to achieve robustness.
Hyperparameter Tuning is All You Need for LISTA
Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unrolling an iterative algorithm and training it like a neural network.
Delayed Propagation Transformer: A Universal Computation Engine towards Practical Control in Cyber-Physical Systems
This paper presents the Delayed Propagation Transformer (DePT), a new transformer-based model that specializes in the global modeling of CPS while taking into account the immutable constraints from the physical world.
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models
To address these pain points, we propose a framework for resource- and parameter-efficient fine-tuning by leveraging the sparsity prior in both weight updates and the final model weights.
You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership
The lottery ticket hypothesis (LTH) emerges as a promising framework to leverage a special sparse subnetwork (i. e., winning ticket) instead of a full model for both training and inference, that can lower both costs without sacrificing the performance.
Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling
Contrastive learning approaches have achieved great success in learning visual representations with few labels of the target classes.
Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods
We propose Cold Brew, a teacher-student distillation approach to address the SCS and noisy-neighbor challenges for GNNs.
Auto-X3D: Ultra-Efficient Video Understanding via Finer-Grained Neural Architecture Search
Efficient video architecture is the key to deploying video recognition systems on devices with limited computing resources.
A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation
In this paper, we comprehensively study three architecture design choices on ViT -- spatial reduction, doubled channels, and multiscale features -- and demonstrate that a vanilla ViT architecture can fulfill this goal without handcrafting multiscale features, maintaining the original ViT design philosophy.
Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better
Federated learning (FL) enables distribution of machine learning workloads from the cloud to resource-limited edge devices.
CADTransformer: Panoptic Symbol Spotting Transformer for CAD Drawings
CADTransformer tokenizes directly from the set of graphical primitives in CAD drawings, and correspondingly optimizes line-grained semantic and instance symbol spotting altogether by a pair of prediction heads.
EI-CLIP: Entity-Aware Interventional Contrastive Learning for E-Commerce Cross-Modal Retrieval
recommendation, and marketing services.
Fast and High-Quality Image Denoising via Malleable Convolutions
These spatially-varying kernels are produced by an efficient predictor network running on a downsampled input, making them much more efficient to compute than per-pixel kernels produced by a full-resolution image, and also enlarging the network's receptive field compared with static kernels.
Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations
Accordingly, we have extended the prefabricated discrete prior in the augmentation set, to a learnable continuous prior in the parameter space of graph generators, assuming that graph priors per se, similar to the concept of image manifolds, can be learned by data generation.
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
To the best of our knowledge, this is the first time quantization has been incorporated into ViT acceleration on FPGAs with the help of a fully automatic framework to guide the quantization strategy on the software side and the accelerator implementations on the hardware side given the target frame rate.
