Search Results for author:
Found 106 papers, 38 papers with code
Min-Max Optimization without Gradients: Convergence and Applications to Black-Box Evasion and Poisoning Attacks
In this paper, we study the problem of constrained min-max optimization in a black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values.
A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction
More and more investors and machine learning models rely on social media (e. g., Twitter and Reddit) to gather real-time information and sentiment to predict stock price movements.
CryoRL: Reinforcement Learning Enables Efficient Cryo-EM Data Collection
Single-particle cryo-electron microscopy (cryo-EM) has become one of the mainstream structural biology techniques because of its ability to determine high-resolution structures of dynamic bio-molecules.
Proactive Image Manipulation Detection
That is, a template protected real image, and its manipulated version, is better discriminated compared to the original real image vs. its manipulated one.
How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective
To tackle this problem, we next propose to prepend an autoencoder (AE) to a given (black-box) model so that DS can be trained using variance-reduced ZO optimization.
Reverse Engineering of Imperceptible Adversarial Image Perturbations
However, carefully crafted, tiny adversarial perturbations are difficult to recover by optimizing a unilateral RED objective.
Optimizer Amalgamation
Selecting an appropriate optimizer for a given problem is of major interest for researchers and practitioners.
Holistic Adversarial Robustness of Deep Learning Models
Adversarial robustness studies the worst-case performance of a machine learning model to ensure safety and reliability.
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited.
To Supervise or Not: How to Effectively Learn Wireless Interference Management Models?
We then provide a series of theoretical results to further understand the properties of the two approaches.
Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization
We first show that the commonly-used Fast-AT is equivalent to using a stochastic gradient algorithm to solve a linearized BLO problem involving a sign operation.
Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework
With the integrated framework, we achieve up to 6\% improvement on the standard accuracy and 17\% improvement on the robust accuracy.
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices.
When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?
We show that AdvCL is able to enhance cross-task robustness transferability without loss of model accuracy and finetuning efficiency.
RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
Specifically, this is the first effort to assign mixed quantization schemes and multiple precisions within layers -- among rows of the DNN weight matrix, for simplified operations in hardware inference, while preserving accuracy.
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works.
An Open Natural Language Processing Development Framework for EHR-based Clinical Research: A case demonstration using the National COVID Cohort Collaborative (N3C)
Although we use COVID-19 as a use case in this effort, our framework is general enough to be applied to other domains of interest in clinical NLP.
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks
Moreover, when the algorithm for training a pruned neural network is specified as an (accelerated) stochastic gradient descent algorithm, we theoretically show that the number of samples required for achieving zero generalization error is proportional to the number of the non-pruned weights in the hidden layer.
Generating Realistic Physical Adversarial Examplesby Patch Transformer Network
Physical adversarial attacks apply carefully crafted adversarial perturbations onto real objects to maliciously alter the prediction of object classifiers or detectors.
Tactics on Refining Decision Boundary for Improving Certification-based Robust Training
In verification-based robust training, existing methods utilize relaxation based methods to bound the worst case performance of neural networks given certain perturbation.
Decentralized Learning for Overparameterized Problems: A Multi-Agent Kernel Approximation Approach
We analyze the optimization and the generalization performance of the proposed framework for the $\ell_2$ loss.
How unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited.
Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD
The key enabling technique is to interpret MAML as a bilevel optimization (BLO) problem and leverage the sign-based SGD(signSGD) as a lower-level optimizer of BLO.
Certifiably Robust Interpretation via Renyi Differential Privacy
The advantages of our Renyi-Robust-Smooth (RDP-based interpretation method) are three-folds.
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Based on our analysis, we summarize a guideline for parameter settings in regards of specific architecture characteristics, which we hope to catalyze the research progress on the topic of lottery ticket hypothesis.
ASK: Adversarial Soft k-Nearest Neighbor Attack and Defense
K-Nearest Neighbor (kNN)-based deep learning methods have been applied to many applications due to their simplicity and geometric interpretability.
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
In this paper, we propose a compression-compilation co-design framework that can guarantee the identified model to meet both resource and real-time specifications of mobile devices.
Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors
In order to reduce the forgetting of particularly earlier learned old knowledge and improve the overall continual learning performance, we propose a simple yet effective fusion mechanism by including all the previously learned feature extractors into the intelligent model.
Preserve, Promote, or Attack? GNN Explanation via Topology Perturbation
Prior works on formalizing explanations of a graph neural network (GNN) focus on a single use case - to preserve the prediction results through identifying important edges and nodes.
Generating Adversarial Computer Programs using Optimized Obfuscations
We further show that our formulation is better at training models that are robust to adversarial attacks.
Adversarial Examples can be Effective Data Augmentation for Unsupervised Machine Learning
In this paper, we propose a framework of generating adversarial examples for unsupervised models and demonstrate novel applications to data augmentation.
On Instabilities of Conventional Multi-Coil MRI Reconstruction to Small Adverserial Perturbations
Although deep learning (DL) has received much attention in accelerated MRI, recent studies suggest small perturbations may lead to instabilities in DL-based reconstructions, leading to concern for their clinical application.
On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning
Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning.
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?
In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i. e., a properly pruned sub-network together with original weight initialization) that can achieve competitive performance than the original dense network.
Fast Training of Provably Robust Neural Networks by SingleProp
Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees.
Robust Overfitting may be mitigated by properly learned smoothening
A recent study (Rice et al., 2020) revealed overfitting to be a dominant phenomenon in adversarially robust training of deep networks, and that appropriate early-stopping of adversarial training (AT) could match the performance gains of most recent algorithmic improvements.
Distributed Adversarial Training to Robustify Deep Neural Networks at Scale
Current deep neural networks are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning
In view of those, we introduce two pruning options, e. g., top-down and bottom-up, for finding lifelong tickets.
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks
Moreover, as the algorithm for training a sparse neural network is specified as (accelerated) stochastic gradient descent algorithm, we theoretically show that the number of samples required for achieving zero generalization error is proportional to the number of the non-pruned model weights in the hidden layer.
Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device
3D object detection is an important task, especially in the autonomous driving application domain.
Self-Progressing Robust Training
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems.
Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework
In this work, we focus on the study of stochastic zeroth-order (ZO) optimization which does not require first-order gradient information and uses only function evaluations.
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
We extend the scope of LTH and question whether matching subnetworks still exist in pre-trained computer vision models, that enjoy the same downstream transfer performance.
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed.
Training Stronger Baselines for Learning to Optimize
Learning to optimize (L2O) has gained increasing attention since classical optimizers require laborious problem-specific design and hyperparameter tuning.
Higher-Order Certification for Randomized Smoothing
We also provide a framework that generalizes the calculation for certification using higher-order information.
TimeAutoML: Autonomous Representation Learning for Multivariate Irregularly Sampled Time Series
To this end, we propose an autonomous representation learning approach for multivariate time series (TimeAutoML) with irregular sampling rates and variable lengths.
Learned Fine-Tuner for Incongruous Few-Shot Adversarial Learning
As novel contributions, we show that the use of LFT within MAML (i) offers the capability to tackle few-shot learning tasks by meta-learning across incongruous yet related problems and (ii) can efficiently work with first-order and derivative-free few-shot learning problems.
Real-Time Execution of Large-scale Language Models on Mobile
Our framework can guarantee the identified model to meet both resource and real-time specifications of mobile devices, thus achieving real-time execution of large transformer-based models like BERT variants.
Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
When the training data are maliciously tampered, the predictions of the acquired deep neural network (DNN) can be manipulated by an adversary known as the Trojan attack (or poisoning backdoor attack).
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
For a range of downstream tasks, we indeed find matching subnetworks at 40% to 90% sparsity.
RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices
The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism.
Proper Network Interpretability Helps Adversarial Robustness in Classification
Recent works have empirically shown that there exist adversarial examples that can be hidden from neural network interpretability (namely, making network interpretation maps visually similar), or interpretability is itself susceptible to adversarial attacks.
Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case
In this paper, we provide a theoretically-grounded generalizability analysis of GNNs with one hidden layer for both regression and binary classification problems.
Can 3D Adversarial Logos Cloak Humans?
Contrary to the traditional adversarial patch, this new form of attack is mapped into the 3D object world and back-propagates to the 2D image domain through differentiable rendering.
Solving Constrained CASH Problems with ADMM
The CASH problem has been widely studied in the context of automated configurations of machine learning (ML) pipelines and various solvers and toolkits are available.
A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning
Zeroth-order (ZO) optimization is a subset of gradient-free optimization that emerges in many signal processing and machine learning applications.
Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
We conduct extensive experiments to demonstrate that the proposed framework achieves large performance margins (eg, 3. 83% on robust accuracy and 1. 3% on standard accuracy, on the CIFAR-10 dataset), compared with the conventional end-to-end adversarial training baseline.
Hidden Cost of Randomized Smoothing
The fragility of modern machine learning models has drawn a considerable amount of attention from both academia and the public.
Defending against Backdoor Attack on Deep Neural Networks
Although deep neural networks (DNNs) have achieved a great success in various computer vision tasks, it is recently found that they are vulnerable to adversarial attacks.
Towards an Efficient and General Framework of Robust Training for Graph Neural Networks
To overcome these limitations, we propose a general framework which leverages the greedy search algorithms and zeroth-order methods to obtain robust GNNs in a generic and an efficient manner.
SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
Structured weight pruning is a representative model compression technique of DNNs for hardware efficiency and inference accelerations.
An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices
Weight pruning has been widely acknowledged as a straightforward and effective method to eliminate redundancy in Deep Neural Networks (DNN), thereby achieving acceleration on various platforms.
Towards Verifying Robustness of Neural Networks Against Semantic Perturbations
Verifying robustness of neural networks given a specified threat model is a fundamental yet challenging task.
Clinical Concept Extraction: a Methodology Review
Background Concept extraction, a subdomain of natural language processing (NLP) with a focus on extracting concepts of interest, has been adopted to computationally extract clinical information from text for a wide range of applications ranging from clinical decision support to care quality improvement.
How can AI Automate End-to-End Data Science?
Data science is labor-intensive and human experts are scarce but heavily involved in every aspect of it.
Adversarial T-shirt! Evading Person Detectors in A Physical World
To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to-rigid objects such as T-shirts.
Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing
Moreover, the same classifier yields the lack of a trade-off with respect to ideal distributions while yielding a trade-off when accuracy is measured with respect to the given (possibly biased) dataset.
ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
In this paper, we propose a zeroth-order AdaMM (ZO-AdaMM) algorithm, that generalizes AdaMM to the gradient-free regime.
Min-Max Optimization without Gradients: Convergence and Applications to Adversarial ML
In this paper, we study the problem of constrained robust (min-max) optimization ina black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values.
Reweighted Proximal Pruning for Large-Scale Language Representation
Is it possible to compress these large-scale language representation models?
Efficient Training of Robust and Verifiable Neural Networks
We propose that many common certified defenses can be viewed under a unified framework of regularization.
Visual Interpretability Alone Helps Adversarial Robustness
Recent works have empirically shown that there exist adversarial examples that can be hidden from neural network interpretability, and interpretability is itself susceptible to adversarial attacks.
Towards A Unified Min-Max Framework for Adversarial Exploration and Robustness
The worst-case training principle that minimizes the maximal adversarial loss, also known as adversarial training (AT), has shown to be a state-of-the-art approach for enhancing adversarial robustness against norm-ball bounded input perturbations.
SPROUT: Self-Progressing Robust Training
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy and reliable machine learning systems.
Sign-OPT: A Query-Efficient Hard-label Adversarial Attack
We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input.
On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method
Robust machine learning is currently one of the most prominent topics which could potentially help shaping a future of advanced AI platforms that not only perform well in average cases but also in worst cases or adverse situations.
Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective
Graph neural networks (GNNs) which apply the deep neural networks to graph data have achieved significant performance for the task of semi-supervised node classification.
Adversarial Attack Generation Empowered by Min-Max Optimization
In this paper, we show how a general framework of min-max optimization over multiple domains can be leveraged to advance the design of different types of adversarial attacks.
Attention Neural Model for Temporal Relation Extraction
Neural network models have shown promise in the temporal relation extraction task.
signSGD via Zeroth-Order Oracle
Our study shows that ZO signSGD requires $\sqrt{d}$ times more iterations than signSGD, leading to a convergence rate of $O(\sqrt{d}/\sqrt{T})$ under mild conditions, where $d$ is the number of optimization variables, and $T$ is the number of iterations.
An ADMM Based Framework for AutoML Pipeline Configuration
We study the AutoML problem of automatically configuring machine learning pipelines by jointly selecting algorithms and their appropriate hyper-parameters for all steps in supervised learning pipelines.
Interpreting Adversarial Examples by Activation Promotion and Suppression
It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers.
Adversarial Robustness vs Model Compression, or Both?
Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting, training a small model from scratch even with inherited initialization from the large model cannot achieve both adversarial robustness and high standard accuracy.
Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates using ADMM
A recent work developed a systematic frame-work of DNN weight pruning using the advanced optimization technique ADMM (Alternating Direction Methods of Multipliers), achieving one of state-of-art in weight pruning results.
CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks
This motivates us to propose a general and efficient framework, CNN-Cert, that is capable of certifying robustness on general convolutional neural networks.
A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM
Both DNN weight pruning and clustering/quantization, as well as their combinations, can be solved in a unified manner.
Progressive Weight Pruning of Deep Neural Networks using ADMM
Motivated by dynamic programming, the proposed method reaches extremely high pruning rate by using partial prunings with moderate pruning rates.
Is Ordered Weighted $\ell_1$ Regularized Regression Robust to Adversarial Perturbation? A Case Study on OSCAR
Many state-of-the-art machine learning models such as deep neural networks have recently shown to be vulnerable to adversarial perturbations, especially in classification tasks.
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
We prove that under our derived conditions, these methods can achieve the convergence rate of order $O(\log{T}/\sqrt{T})$ for nonconvex stochastic optimization.
Structured Adversarial Attack: Towards General Implementation and Better Interpretability
When generating adversarial examples to attack deep neural networks (DNNs), Lp norm of the added perturbation is usually used to measure the similarity between original image and adversarial example.
Latent heterogeneous multilayer community detection
We propose a method for simultaneously detecting shared and unshared communities in heterogeneous multilayer weighted and undirected networks.
AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting.
Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications
The von Neumann graph entropy (VNGE) facilitates measurement of information divergence and distance between graphs in a graph sequence.
Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization
As application demands for zeroth-order (gradient-free) optimization accelerate, the need for variance reduced and faster converging approaches is also intensifying.
An ADMM-Based Universal Framework for Adversarial Attacks on Deep Neural Networks
In the literature, the added distortions are usually measured by L0, L1, L2, and L infinity norms, namely, L0, L1, L2, and L infinity attacks, respectively.
A Comparison of Word Embeddings for the Biomedical Natural Language Processing
First, the word embeddings trained on clinical notes and biomedical publications can capture the semantics of medical terms better, and find more relevant similar medical terms, and are closer to human experts' judgments, compared to these trained on Wikipedia and news.
Information Retrieval
A New Data-Driven Sparse-Learning Approach to Study Chemical Reaction Networks
In this paper, we introduce a novel approach for the identification of the influential reactions in chemical reaction networks for combustion applications, using a data-driven sparse-learning technique.
A Data-Driven Sparse-Learning Approach to Model Reduction in Chemical Reaction Networks
In this paper, we propose an optimization-based sparse learning approach to identify the set of most influential reactions in a chemical reaction network.
Semiblind subgraph reconstruction in Gaussian graphical models
Consider a social network where only a few nodes (agents) have meaningful interactions in the sense that the conditional dependency graph over node attribute variables (behaviors) is sparse.
Zeroth-Order Online Alternating Direction Method of Multipliers: Convergence Analysis and Applications
In this paper, we design and analyze a new zeroth-order online algorithm, namely, the zeroth-order online alternating direction method of multipliers (ZOO-ADMM), which enjoys dual advantages of being gradient-free operation and employing the ADMM to accommodate complex structured regularizers.
A Memristor-Based Optimization Framework for AI Applications
In addition to ADMM, implementation of a customized power iteration (PI) method for eigenvalue/eigenvector computation using memristor crossbars is discussed.
MayoNLP at SemEval 2017 Task 10: Word Embedding Distance Pattern for Keyphrase Classification in Scientific Publications
We explored semantic similarities and patterns of keyphrases in scientific publications using pre-trained word embedding models.
Bias-Variance Tradeoff of Graph Laplacian Regularizer
This paper presents a bias-variance tradeoff of graph Laplacian regularizer, which is widely used in graph signal processing and semi-supervised learning tasks.
Accelerated Distributed Dual Averaging over Evolving Networks of Growing Connectivity
Our analysis reveals the connection between network topology design and the convergence rate of DDA, and provides quantitative evaluation of DDA acceleration for distributed optimization that is absent in the existing analysis.
Learning Sparse Graphs Under Smoothness Prior
Given the noisy data, we show that the joint sparse graph learning and denoising problem can be simplified to designing only the sparse edge selection vector, which can be solved using convex optimization.
