Search Results for author:
Found 146 papers, 59 papers with code
Min-Max Optimization without Gradients: Convergence and Applications to Black-Box Evasion and Poisoning Attacks
In this paper, we study the problem of constrained min-max optimization in a black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values.
Robust Mixture-of-Expert Training for Convolutional Neural Networks
Since the lack of robustness has become one of the main hurdles for CNNs, in this paper we ask: How to adversarially robustify a CNN-based MoE model?
Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks
Backward propagation (BP) is widely used to compute the gradients in neural network training.
AutoSeqRec: Autoencoder for Efficient Sequential Recommendation
Sequential recommendation demonstrates the capability to recommend items by modeling the sequential behavior of users.
An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning
Overall, we hope that this article can serve to accelerate the adoption of BLO as a generic tool to model, analyze, and innovate on a wide array of emerging SP and ML applications.
Certified Robustness for Large Language Models with Self-Denoising
This largely falls into the study of certified robust LLMs, i. e., all predictions of LLM are certified to be correct in a local region around the input.
Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations
To mitigate the gap, we designed DeepFuse, the first interactive design that realizes the direct feedback loop between a user and CNNs in diagnosing and revising CNN's vulnerability using local explanations.
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks
In deep learning, mixture-of-experts (MoE) activates one or few experts (sub-networks) on a per-sample or per-token basis, resulting in significant computation reduction.
Model Sparsity Can Simplify Machine Unlearning
We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient.
A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion
In this work, we study the problem of adversarial attack generation for Stable Diffusion and ask if an adversarial text prompt can be obtained even in the absence of end-to-end model queries.
Fairness Improves Learning from Noisily Labeled Long-Tailed Data
Both long-tailed and noisily labeled data frequently appear in real-world applications and impose significant challenges for learning.
Robust Mode Connectivity-Oriented Adversarial Defense: Enhancing Neural Network Robustness Against Diversified $\ell_p$ Attacks
Adversarial robustness is a key concept in measuring the ability of neural networks to defend against adversarial attacks during the inference phase.
SMUG: Towards robust MRI reconstruction by smoothed unrolling
To address this problem, we propose a novel image reconstruction framework, termed SMOOTHED UNROLLING (SMUG), which advances a deep unrolling-based MRI reconstruction model using a randomized smoothing (RS)-based robust learning operation.
Can Adversarial Examples Be Parsed to Reveal Victim Model Information?
We call this 'model parsing of adversarial attacks' - a task to uncover 'arcana' in terms of the concealed VM information in attacks.
Text-Visual Prompting for Efficient 2D Temporal Video Grounding
In this paper, we study the problem of temporal video grounding (TVG), which aims to predict the starting/ending time points of moments described by a text sentence within a long untrimmed video.
Robustness-preserving Lifelong Learning via Dataset Condensation
Most work in this learning paradigm has focused on resolving the problem of 'catastrophic forgetting,' which refers to a notorious dilemma between improving model accuracy over new data and retaining accuracy over previous data.
What Is Missing in IRM Training and Evaluation? Challenges and Solutions
We propose a new IRM variant to address this limitation based on a novel viewpoint of ensemble IRM games as consensus-constrained bi-level optimization.
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity
Based on a data model characterizing both label-relevant and label-irrelevant tokens, this paper provides the first theoretical analysis of training a shallow ViT, i. e., one self-attention layer followed by a two-layer perceptron, for a classification task.
Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks
Due to the significant computational challenge of training large-scale graph neural networks (GNNs), various sparse learning techniques have been exploited to reduce memory and storage costs.
Certified Interpretability Robustness for Class Activation Mapping
Interpreting machine learning models is challenging but crucial for ensuring the safety of deep networks in autonomous driving systems.
Towards Understanding How Self-training Tolerates Data Backdoor Poisoning
In this paper, we explore the potential of self-training via additional unlabeled data for mitigating backdoor attacks.
Adaptively Integrated Knowledge Distillation and Prediction Uncertainty for Continual Learning
The first one is to adaptively integrate multiple levels of old knowledge and transfer it to each block level in the new model.
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines
These guidelines provide information about the context they are applicable to and what should be included in the response, allowing the models to generate responses that are more closely aligned with the developer's expectations and intent.
TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimization
Robustness evaluation against adversarial examples has become increasingly important to unveil the trustworthiness of the prevailing deep models in natural language processing (NLP).
Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization
In this paper, we design and analyze stochastic inexact augmented Lagrangian methods (Stoc-iALM) to solve problems involving a nonconvex composite (i. e. smooth+nonsmooth) objective and nonconvex smooth functional constraints.
CLAWSAT: Towards Both Robust and Accurate Code Models
We integrate contrastive learning (CL) with adversarial learning to co-optimize the robustness and accuracy of code models.
Understanding and Improving Visual Prompting: A Label-Mapping Perspective
As highlighted below, we show that when reprogramming an ImageNet-pretrained ResNet-18 to 13 target tasks, our method outperforms baselines by a substantial margin, e. g., 7. 9% and 6. 7% accuracy improvements in transfer learning to the target Flowers102 and CIFAR100 datasets.
On the Robustness of deep learning-based MRI Reconstruction to image transformations
We find a new instability source of MRI image reconstruction, i. e., the lack of reconstruction robustness against spatial transformations of an input, e. g., rotation and cutout.
Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices
In this paper, we propose a data-model-hardware tri-design framework for high-throughput, low-cost, and high-accuracy multi-object tracking (MOT) on High-Definition (HD) video stream.
Visual Prompting for Adversarial Robustness
In this work, we leverage visual prompting (VP) to improve adversarial robustness of a fixed, pre-trained model at testing time.
Advancing Model Pruning via Bi-level Optimization
To reduce the computation overhead, various efficient 'one-shot' pruning methods have been developed, but these schemes are usually unable to find winning tickets as good as IMP.
Fairness Reprogramming
Specifically, FairReprogram considers the case where models can not be changed and appends to the input a set of perturbations, called the fairness trigger, which is tuned towards the fairness criteria under a min-max formulation.
Saliency Guided Adversarial Training for Learning Generalizable Features with Applications to Medical Imaging Classification System
We hypothesize that adversarial training can eliminate shortcut features whereas saliency guided training can filter out non-relevant features; both are nuisance features accounting for the performance degradation on OOD test sets.
Improving Bot Response Contradiction Detection via Utterance Rewriting
Previous work has treated contradiction detection in bot responses as a task similar to natural language inference, e. g., detect the contradiction between a pair of bot utterances.
Generalization Guarantee of Training Graph Convolutional Networks with Graph Topology Sampling
Graph convolutional networks (GCNs) have recently achieved great empirical success in learning graph-structured data.
The NLP Sandbox: an efficient model-to-data system to enable federated and unbiased evaluation of clinical NLP models
Discussion We demonstrated the feasibility of using the NLP Sandbox to conduct a multi-site evaluation of clinical text de-identification models without the sharing of data.
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning
Inspired by the recent success of learning robust models with unlabeled data, we explore a new robustness-aware CIL setting, where the learned adversarial robustness has to resist forgetting and be transferred as new tasks come in continually.
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness
Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish.
Distributed Adversarial Training to Robustify Deep Neural Networks at Scale
Spurred by that, we propose distributed adversarial training (DAT), a large-batch adversarial training framework implemented over multiple machines.
Data-Efficient Double-Win Lottery Tickets from Robust Pre-training
For example, on downstream CIFAR-10/100 datasets, we identify double-win matching subnetworks with the standard, fast adversarial, and adversarial pre-training from ImageNet, at 89. 26%/73. 79%, 89. 26%/79. 03%, and 91. 41%/83. 22% sparsity, respectively.
Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning
Existing knowledge integration approaches are limited to using differentiable knowledge source to be compatible with first-order DL training paradigm.
Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free
Trojan attacks threaten deep neural networks (DNNs) by poisoning them to behave normally on most samples, yet to produce manipulated results for inputs attached with a particular trigger.
A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Predictions
More and more investors and machine learning models rely on social media (e. g., Twitter and Reddit) to gather real-time information and sentiment to predict stock price movements.
CryoRL: Reinforcement Learning Enables Efficient Cryo-EM Data Collection
Single-particle cryo-electron microscopy (cryo-EM) has become one of the mainstream structural biology techniques because of its ability to determine high-resolution structures of dynamic bio-molecules.
Proactive Image Manipulation Detection
That is, a template protected real image, and its manipulated version, is better discriminated compared to the original real image vs. its manipulated one.
How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective
To tackle this problem, we next propose to prepend an autoencoder (AE) to a given (black-box) model so that DS can be trained using variance-reduced ZO optimization.
Reverse Engineering of Imperceptible Adversarial Image Perturbations
However, carefully crafted, tiny adversarial perturbations are difficult to recover by optimizing a unilateral RED objective.
Optimizer Amalgamation
Selecting an appropriate optimizer for a given problem is of major interest for researchers and practitioners.
Holistic Adversarial Robustness of Deep Learning Models
Adversarial robustness studies the worst-case performance of a machine learning model to ensure safety and reliability.
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited.
To Supervise or Not: How to Effectively Learn Wireless Interference Management Models?
We then provide a series of theoretical results to further understand the properties of the two approaches.
Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization
We first show that the commonly-used Fast-AT is equivalent to using a stochastic gradient algorithm to solve a linearized BLO problem involving a sign operation.
Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework
With the integrated framework, we achieve up to 6\% improvement on the standard accuracy and 17\% improvement on the robust accuracy.
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices.
When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?
We show that AdvCL is able to enhance cross-task robustness transferability without loss of model accuracy and finetuning efficiency.
RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
Specifically, this is the first effort to assign mixed quantization schemes and multiple precisions within layers -- among rows of the DNN weight matrix, for simplified operations in hardware inference, while preserving accuracy.
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works.
An Open Natural Language Processing Development Framework for EHR-based Clinical Research: A case demonstration using the National COVID Cohort Collaborative (N3C)
Although we use COVID-19 as a use case in this effort, our framework is general enough to be applied to other domains of interest in clinical NLP.
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks
Moreover, when the algorithm for training a pruned neural network is specified as an (accelerated) stochastic gradient descent algorithm, we theoretically show that the number of samples required for achieving zero generalization error is proportional to the number of the non-pruned weights in the hidden layer.
Decentralized Learning for Overparameterized Problems: A Multi-Agent Kernel Approximation Approach
We analyze the optimization and the generalization performance of the proposed framework for the $\ell_2$ loss.
Generating Realistic Physical Adversarial Examplesby Patch Transformer Network
Physical adversarial attacks apply carefully crafted adversarial perturbations onto real objects to maliciously alter the prediction of object classifiers or detectors.
Tactics on Refining Decision Boundary for Improving Certification-based Robust Training
In verification-based robust training, existing methods utilize relaxation based methods to bound the worst case performance of neural networks given certain perturbation.
How unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited.
Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD
The key enabling technique is to interpret MAML as a bilevel optimization (BLO) problem and leverage the sign-based SGD(signSGD) as a lower-level optimizer of BLO.
Certifiably Robust Interpretation via Renyi Differential Privacy
The advantages of our Renyi-Robust-Smooth (RDP-based interpretation method) are three-folds.
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Based on our analysis, we summarize a guideline for parameter settings in regards of specific architecture characteristics, which we hope to catalyze the research progress on the topic of lottery ticket hypothesis.
ASK: Adversarial Soft k-Nearest Neighbor Attack and Defense
K-Nearest Neighbor (kNN)-based deep learning methods have been applied to many applications due to their simplicity and geometric interpretability.
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
In this paper, we propose a compression-compilation co-design framework that can guarantee the identified model to meet both resource and real-time specifications of mobile devices.
Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors
In order to reduce the forgetting of particularly earlier learned old knowledge and improve the overall continual learning performance, we propose a simple yet effective fusion mechanism by including all the previously learned feature extractors into the intelligent model.
Preserve, Promote, or Attack? GNN Explanation via Topology Perturbation
Prior works on formalizing explanations of a graph neural network (GNN) focus on a single use case - to preserve the prediction results through identifying important edges and nodes.
Generating Adversarial Computer Programs using Optimized Obfuscations
We further show that our formulation is better at training models that are robust to adversarial attacks.
Adversarial Examples can be Effective Data Augmentation for Unsupervised Machine Learning
In this paper, we propose a framework of generating adversarial examples for unsupervised models and demonstrate novel applications to data augmentation.
On Instabilities of Conventional Multi-Coil MRI Reconstruction to Small Adverserial Perturbations
Although deep learning (DL) has received much attention in accelerated MRI, recent studies suggest small perturbations may lead to instabilities in DL-based reconstructions, leading to concern for their clinical application.
On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning
Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning.
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?
In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i. e., a properly pruned sub-network together with original weight initialization) that can achieve competitive performance than the original dense network.
Fast Training of Provably Robust Neural Networks by SingleProp
Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees.
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks
Moreover, as the algorithm for training a sparse neural network is specified as (accelerated) stochastic gradient descent algorithm, we theoretically show that the number of samples required for achieving zero generalization error is proportional to the number of the non-pruned model weights in the hidden layer.
Robust Overfitting may be mitigated by properly learned smoothening
A recent study (Rice et al., 2020) revealed overfitting to be a dominant phenomenon in adversarially robust training of deep networks, and that appropriate early-stopping of adversarial training (AT) could match the performance gains of most recent algorithmic improvements.
Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning
In view of those, we introduce two pruning options, e. g., top-down and bottom-up, for finding lifelong tickets.
Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device
3D object detection is an important task, especially in the autonomous driving application domain.
Self-Progressing Robust Training
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems.
Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework
In this work, we focus on the study of stochastic zeroth-order (ZO) optimization which does not require first-order gradient information and uses only function evaluations.
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
We extend the scope of LTH and question whether matching subnetworks still exist in pre-trained computer vision models, that enjoy the same downstream transfer performance.
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed.
Training Stronger Baselines for Learning to Optimize
Learning to optimize (L2O) has gained increasing attention since classical optimizers require laborious problem-specific design and hyperparameter tuning.
Higher-Order Certification for Randomized Smoothing
We also provide a framework that generalizes the calculation for certification using higher-order information.
TimeAutoML: Autonomous Representation Learning for Multivariate Irregularly Sampled Time Series
To this end, we propose an autonomous representation learning approach for multivariate time series (TimeAutoML) with irregular sampling rates and variable lengths.
Learning to Generate Image Source-Agnostic Universal Adversarial Perturbations
The resulting scheme for meta-learning a UAP generator (i) has better performance (50% higher ASR) than baselines such as Projected Gradient Descent, (ii) has better performance (37% faster) than the vanilla L2O and MAML frameworks (when applicable), and (iii) is able to simultaneously handle UAP generation for different victim models and image data sources.
Real-Time Execution of Large-scale Language Models on Mobile
Our framework can guarantee the identified model to meet both resource and real-time specifications of mobile devices, thus achieving real-time execution of large transformer-based models like BERT variants.
Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
When the training data are maliciously tampered, the predictions of the acquired deep neural network (DNN) can be manipulated by an adversary known as the Trojan attack (or poisoning backdoor attack).
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
For a range of downstream tasks, we indeed find matching subnetworks at 40% to 90% sparsity.
RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices
The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism.
Proper Network Interpretability Helps Adversarial Robustness in Classification
Recent works have empirically shown that there exist adversarial examples that can be hidden from neural network interpretability (namely, making network interpretation maps visually similar), or interpretability is itself susceptible to adversarial attacks.
Can 3D Adversarial Logos Cloak Humans?
Contrary to the traditional adversarial patch, this new form of attack is mapped into the 3D object world and back-propagates to the 2D image domain through differentiable rendering.
Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case
In this paper, we provide a theoretically-grounded generalizability analysis of GNNs with one hidden layer for both regression and binary classification problems.
Solving Constrained CASH Problems with ADMM
The CASH problem has been widely studied in the context of automated configurations of machine learning (ML) pipelines and various solvers and toolkits are available.
A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning
Zeroth-order (ZO) optimization is a subset of gradient-free optimization that emerges in many signal processing and machine learning applications.
Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
We conduct extensive experiments to demonstrate that the proposed framework achieves large performance margins (eg, 3. 83% on robust accuracy and 1. 3% on standard accuracy, on the CIFAR-10 dataset), compared with the conventional end-to-end adversarial training baseline.
Hidden Cost of Randomized Smoothing
The fragility of modern machine learning models has drawn a considerable amount of attention from both academia and the public.
Defending against Backdoor Attack on Deep Neural Networks
Although deep neural networks (DNNs) have achieved a great success in various computer vision tasks, it is recently found that they are vulnerable to adversarial attacks.
Towards an Efficient and General Framework of Robust Training for Graph Neural Networks
To overcome these limitations, we propose a general framework which leverages the greedy search algorithms and zeroth-order methods to obtain robust GNNs in a generic and an efficient manner.
SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
Structured weight pruning is a representative model compression technique of DNNs for hardware efficiency and inference accelerations.
An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices
Weight pruning has been widely acknowledged as a straightforward and effective method to eliminate redundancy in Deep Neural Networks (DNN), thereby achieving acceleration on various platforms.
Towards Verifying Robustness of Neural Networks Against Semantic Perturbations
Verifying robustness of neural networks given a specified threat model is a fundamental yet challenging task.
Clinical Concept Extraction: a Methodology Review
Background Concept extraction, a subdomain of natural language processing (NLP) with a focus on extracting concepts of interest, has been adopted to computationally extract clinical information from text for a wide range of applications ranging from clinical decision support to care quality improvement.
How can AI Automate End-to-End Data Science?
Data science is labor-intensive and human experts are scarce but heavily involved in every aspect of it.
Adversarial T-shirt! Evading Person Detectors in A Physical World
To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to-rigid objects such as T-shirts.
Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing
Moreover, the same classifier yields the lack of a trade-off with respect to ideal distributions while yielding a trade-off when accuracy is measured with respect to the given (possibly biased) dataset.
ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
In this paper, we propose a zeroth-order AdaMM (ZO-AdaMM) algorithm, that generalizes AdaMM to the gradient-free regime.
Min-Max Optimization without Gradients: Convergence and Applications to Adversarial ML
In this paper, we study the problem of constrained robust (min-max) optimization ina black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values.
Reweighted Proximal Pruning for Large-Scale Language Representation
Is it possible to compress these large-scale language representation models?
Towards A Unified Min-Max Framework for Adversarial Exploration and Robustness
The worst-case training principle that minimizes the maximal adversarial loss, also known as adversarial training (AT), has shown to be a state-of-the-art approach for enhancing adversarial robustness against norm-ball bounded input perturbations.
Efficient Training of Robust and Verifiable Neural Networks
We propose that many common certified defenses can be viewed under a unified framework of regularization.
Visual Interpretability Alone Helps Adversarial Robustness
Recent works have empirically shown that there exist adversarial examples that can be hidden from neural network interpretability, and interpretability is itself susceptible to adversarial attacks.
SPROUT: Self-Progressing Robust Training
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy and reliable machine learning systems.
Sign-OPT: A Query-Efficient Hard-label Adversarial Attack
We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input.
On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method
Robust machine learning is currently one of the most prominent topics which could potentially help shaping a future of advanced AI platforms that not only perform well in average cases but also in worst cases or adverse situations.
Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective
Graph neural networks (GNNs) which apply the deep neural networks to graph data have achieved significant performance for the task of semi-supervised node classification.
Adversarial Attack Generation Empowered by Min-Max Optimization
In this paper, we show how a general framework of min-max optimization over multiple domains can be leveraged to advance the design of different types of adversarial attacks.
Attention Neural Model for Temporal Relation Extraction
Neural network models have shown promise in the temporal relation extraction task.
signSGD via Zeroth-Order Oracle
Our study shows that ZO signSGD requires $\sqrt{d}$ times more iterations than signSGD, leading to a convergence rate of $O(\sqrt{d}/\sqrt{T})$ under mild conditions, where $d$ is the number of optimization variables, and $T$ is the number of iterations.
An ADMM Based Framework for AutoML Pipeline Configuration
We study the AutoML problem of automatically configuring machine learning pipelines by jointly selecting algorithms and their appropriate hyper-parameters for all steps in supervised learning pipelines.
Interpreting Adversarial Examples by Activation Promotion and Suppression
It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers.
Adversarial Robustness vs Model Compression, or Both?
Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting, training a small model from scratch even with inherited initialization from the large model cannot achieve both adversarial robustness and high standard accuracy.
Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates using ADMM
A recent work developed a systematic frame-work of DNN weight pruning using the advanced optimization technique ADMM (Alternating Direction Methods of Multipliers), achieving one of state-of-art in weight pruning results.
CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks
This motivates us to propose a general and efficient framework, CNN-Cert, that is capable of certifying robustness on general convolutional neural networks.
A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM
Both DNN weight pruning and clustering/quantization, as well as their combinations, can be solved in a unified manner.
Progressive Weight Pruning of Deep Neural Networks using ADMM
Motivated by dynamic programming, the proposed method reaches extremely high pruning rate by using partial prunings with moderate pruning rates.
Is Ordered Weighted $\ell_1$ Regularized Regression Robust to Adversarial Perturbation? A Case Study on OSCAR
Many state-of-the-art machine learning models such as deep neural networks have recently shown to be vulnerable to adversarial perturbations, especially in classification tasks.
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
We prove that under our derived conditions, these methods can achieve the convergence rate of order $O(\log{T}/\sqrt{T})$ for nonconvex stochastic optimization.
Structured Adversarial Attack: Towards General Implementation and Better Interpretability
When generating adversarial examples to attack deep neural networks (DNNs), Lp norm of the added perturbation is usually used to measure the similarity between original image and adversarial example.
Latent heterogeneous multilayer community detection
We propose a method for simultaneously detecting shared and unshared communities in heterogeneous multilayer weighted and undirected networks.
Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications
The von Neumann graph entropy (VNGE) facilitates measurement of information divergence and distance between graphs in a graph sequence.
AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting.
Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization
As application demands for zeroth-order (gradient-free) optimization accelerate, the need for variance reduced and faster converging approaches is also intensifying.
An ADMM-Based Universal Framework for Adversarial Attacks on Deep Neural Networks
In the literature, the added distortions are usually measured by L0, L1, L2, and L infinity norms, namely, L0, L1, L2, and L infinity attacks, respectively.
A Comparison of Word Embeddings for the Biomedical Natural Language Processing
First, the word embeddings trained on clinical notes and biomedical publications can capture the semantics of medical terms better, and find more relevant similar medical terms, and are closer to human experts' judgments, compared to these trained on Wikipedia and news.
Information Retrieval
A New Data-Driven Sparse-Learning Approach to Study Chemical Reaction Networks
In this paper, we introduce a novel approach for the identification of the influential reactions in chemical reaction networks for combustion applications, using a data-driven sparse-learning technique.
A Data-Driven Sparse-Learning Approach to Model Reduction in Chemical Reaction Networks
In this paper, we propose an optimization-based sparse learning approach to identify the set of most influential reactions in a chemical reaction network.
Semiblind subgraph reconstruction in Gaussian graphical models
Consider a social network where only a few nodes (agents) have meaningful interactions in the sense that the conditional dependency graph over node attribute variables (behaviors) is sparse.
Zeroth-Order Online Alternating Direction Method of Multipliers: Convergence Analysis and Applications
In this paper, we design and analyze a new zeroth-order online algorithm, namely, the zeroth-order online alternating direction method of multipliers (ZOO-ADMM), which enjoys dual advantages of being gradient-free operation and employing the ADMM to accommodate complex structured regularizers.
A Memristor-Based Optimization Framework for AI Applications
In addition to ADMM, implementation of a customized power iteration (PI) method for eigenvalue/eigenvector computation using memristor crossbars is discussed.
MayoNLP at SemEval 2017 Task 10: Word Embedding Distance Pattern for Keyphrase Classification in Scientific Publications
We explored semantic similarities and patterns of keyphrases in scientific publications using pre-trained word embedding models.
Bias-Variance Tradeoff of Graph Laplacian Regularizer
This paper presents a bias-variance tradeoff of graph Laplacian regularizer, which is widely used in graph signal processing and semi-supervised learning tasks.
Accelerated Distributed Dual Averaging over Evolving Networks of Growing Connectivity
Our analysis reveals the connection between network topology design and the convergence rate of DDA, and provides quantitative evaluation of DDA acceleration for distributed optimization that is absent in the existing analysis.
Learning Sparse Graphs Under Smoothness Prior
Given the noisy data, we show that the joint sparse graph learning and denoising problem can be simplified to designing only the sparse edge selection vector, which can be solved using convex optimization.
