Search Results for author:
Found 194 papers, 88 papers with code
On the Transferability of Adversarial Attacks against Neural Text Classifier
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
On Lp-norm Robustness of Ensemble Decision Stumps and Trees
In this paper, we study the robustness verification and defense with respect to general $\ell_p$ norm perturbation for ensemble trees and stumps.
Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples
Most of the existing defense methods improve the adversarial robustness by making the models adapt to the training set augmented with some adversarial examples.
Online Continuous Hyperparameter Optimization for Contextual Bandits
Specifically, we use a double-layer bandit framework named CDT (Continuous Dynamic Tuning) and formulate the hyperparameter optimization as a non-stationary continuum-armed bandit, where each arm represents a combination of hyperparameters, and the corresponding reward is the algorithmic result.
Symbolic Discovery of Optimization Algorithms
We present a method to formulate algorithm discovery as program search, and apply it to discover optimization algorithms for deep neural network training.
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data
In this paper, we propose a new effective robustness evaluation metric to compare the effective robustness of models trained on different data distributions.
Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory
Among recently proposed methods, Matching Training Trajectories (MTT) achieves state-of-the-art performance on CIFAR-10/100, while having difficulty scaling to ImageNet-1k dataset due to the large memory requirement when performing unrolled gradient computation through back-propagation.
Are AlphaZero-like Agents Robust to Adversarial Perturbations?
Given that the state space of Go is extremely large and a human player can play the game from any legal state, we ask whether adversarial states exist for Go AIs that may lead them to play surprisingly wrong actions.
Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning
Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks.
Preserving In-Context Learning ability in Large Language Model Fine-tuning
More importantly, ProMoT shows remarkable generalization ability on tasks that have different formats, e. g. fine-tuning on a NLI binary classification task improves the model's in-context ability to do summarization (+0. 53 Rouge-2 score compared to the pretrained model), making ProMoT a promising method to build general purpose capabilities such as grounding and reasoning into LLMs with small but high quality datasets.
ADDMU: Detection of Far-Boundary Adversarial Examples with Data and Model Uncertainty Estimation
Finally, our analysis shows that the two types of uncertainty provided by \textbf{ADDMU} can be leveraged to characterize adversarial examples and identify the ones that contribute most to model's robustness in adversarial training.
Reducing Training Sample Memorization in GANs by Training with Memorization Rejection
In this paper, we propose memorization rejection, a training scheme that rejects generated samples that are near-duplicates of training samples during training.
Uncertainty in Extreme Multi-label Classification
Uncertainty quantification is one of the most crucial tasks to obtain trustworthy and reliable machine learning models for decision making.
ELIAS: End-to-End Learning to Index and Search in Large Output Spaces
A popular approach for dealing with the large label space is to arrange the labels into a shallow tree-based index and then learn an ML model to efficiently search this index via beam search.
Watermarking Pre-trained Language Models with Backdooring
Large pre-trained language models (PLMs) have proven to be a crucial component of modern natural language processing systems.
Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagation
In this paper, we develop an efficient framework for computing the $\ell_\infty$ local Lipschitz constant of a neural network by tightly upper bounding the norm of Clarke Jacobian via linear bound propagation.
Efficient Non-Parametric Optimizer Search for Diverse Tasks
Efficient and automated design of optimizers plays a crucial role in full-stack AutoML systems.
Concept Gradient: Concept-based Interpretation Without Linear Assumption
We showed that for a general (potentially non-linear) concept, we can mathematically evaluate how a small change of concept affecting the model's prediction, which leads to an extension of gradient-based interpretation to the concept space.
General Cutting Planes for Bound-Propagation-Based Neural Network Verification
Our generalized bound propagation method, GCP-CROWN, opens up the opportunity to apply general cutting plane methods for neural network verification while benefiting from the efficiency and GPU acceleration of bound propagation methods.
FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning
Federated learning~(FL) has recently attracted increasing attention from academia and industry, with the ultimate goal of achieving collaborative training under privacy and communication constraints.
DC-BENCH: Dataset Condensation Benchmark
Dataset Condensation is a newly emerging technique aiming at learning a tiny dataset that captures the rich information encoded in the original dataset.
Improving the Adversarial Robustness of NLP Models by Information Bottleneck
Existing studies have demonstrated that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive, but can be easily manipulated by adversaries to fool NLP models.
Generalizing Few-Shot NAS with Gradient Matching
Efficient performance estimation of architectures drawn from large search spaces is essential to Neural Architecture Search.
On the Convergence of Certified Robust Training with Interval Bound Propagation
Interval Bound Propagation (IBP) is so far the base of state-of-the-art methods for training neural networks with certifiable robustness guarantees when potential adversarial perturbations present, while the convergence of IBP training remains unknown in existing literature.
Towards Efficient and Scalable Sharpness-Aware Minimization
Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of the loss landscape and generalization, has demonstrated significant performance boosts on training large-scale models such as vision transformers.
Extreme Zero-Shot Learning for Extreme Text Classification
To learn the semantic embeddings of instances and labels with raw text, we propose to pre-train Transformer-based encoders with self-supervised contrastive losses.
Multi Label Text Classification
Multi-Label Text Classification
+2
Temporal Shuffling for Defending Deep Action Recognition Models against Adversarial Attacks
Another observation enabling our defense method is that adversarial perturbations on videos are sensitive to temporal destruction.
DRONE: Data-aware Low-rank Compression for Large NLP Models
In addition to compressing standard models, out method can also be used on distilled BERT models to further improve compression rate.
A Review of Adversarial Attack and Defense for Classification Methods
Despite the efficiency and scalability of machine learning systems, recent studies have demonstrated that many classification methods, especially deep neural networks (DNNs), are vulnerable to adversarial examples; i. e., examples that are carefully crafted to fool a well-trained classification model while being indistinguishable from natural data to human.
Can Vision Transformers Perform Convolution?
Several recent studies have demonstrated that attention-based networks, such as Vision Transformer (ViT), can outperform Convolutional Neural Networks (CNNs) on several computer vision tasks without using convolutional layers.
Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction
We also provide a theoretical analysis that justifies the use of XMC over link prediction and motivates integrating XR-Transformers, a powerful method for solving XMC problems, into the GIANT framework.
Ranked #2 on
Node Property Prediction
on ogbn-papers100M
How and When Adversarial Robustness Transfers in Knowledge Distillation?
Our comprehensive analysis shows several novel insights that (1) With KDIGA, students can preserve or even exceed the adversarial robustness of the teacher model, even when their models have fundamentally different architectures; (2) KDIGA enables robustness to transfer to pre-trained students, such as KD from an adversarially trained ResNet to a pre-trained ViT, without loss of clean accuracy; and (3) Our derived local linearity bounds for characterizing adversarial robustness in KD are consistent with the empirical results.
Adversarial Attack across Datasets
In this paper, we define a Generalized Transferable Attack (GTA) problem where the attacker doesn't know this information and is acquired to attack any randomly encountered images that may come from unknown datasets.
Learning to Schedule Learning rate with Graph Neural Networks
By constructing a directed graph for the underlying neural network of the target problem, GNS encodes current dynamics with a graph message passing network and trains an agent to control the learning rate accordingly via reinforcement learning.
A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks
In this work, we formulate an adversarial attack using a branch-and-bound (BaB) procedure on ReLU neural networks and search adversarial examples in the activation space corresponding to binary variables in a mixed integer programming (MIP) formulation.
Sharpness-Aware Minimization in Large-Batch Training: Training Vision Transformer In Minutes
Large-batch training is an important direction for distributed machine learning, which can improve the utilization of large-scale clusters and therefore accelerate the training process.
FastEnsemble: Benchmarking and Accelerating Ensemble-based Uncertainty Estimation for Image-to-Image Translation
In addition, we introduce FastEnsemble, a fast ensemble method which only requires less than $8\%$ of the full-ensemble training time to generate a new ensemble member.
Training Meta-Surrogate Model for Transferable Adversarial Attack
In this paper, we tackle this problem from a novel angle -- instead of using the original surrogate models, can we obtain a Meta-Surrogate Model (MSM) such that attacks to this model can be easier transferred to other models?
Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution
Recent studies have shown that deep neural networks are vulnerable to intentionally crafted adversarial examples, and various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models.
RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving
Predictor-based algorithms have achieved remarkable performance in the Neural Architecture Search (NAS) tasks.
RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection
In particular, we propose to generate random layouts of a scene by making use of the objects in the synthetic CAD dataset and learn the 3D scene representation by applying object-level contrastive learning on two random scenes generated from the same set of synthetic objects.
Rethinking Architecture Selection in Differentiable NAS
Differentiable Neural Architecture Search is one of the most popular Neural Architecture Search (NAS) methods for its search efficiency and simplicity, accomplished by jointly optimizing the model weight and architecture parameters in a weight-sharing supernet via gradient-based algorithms.
Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble
Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples.
Label Disentanglement in Partition-based Extreme Multilabel Classification
Partition-based methods are increasingly-used in extreme multi-label classification (XMC) problems due to their scalability to large output spaces (e. g., millions or more).
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification
We develop $\beta$-CROWN, a new bound propagation based method that can fully encode neuron split constraints in branch-and-bound (BaB) based complete verification via optimizable parameters $\beta$.
Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
Attentional mechanisms are order-invariant.
Robust Stochastic Linear Contextual Bandits Under Adversarial Attacks
We provide theoretical guarantees for our proposed algorithm and show by experiments that our proposed algorithm improves the robustness against various kinds of popular attacks.
Syndicated Bandits: A Framework for Auto Tuning Hyper-parameters in Contextual Bandit Algorithms
To tackle this problem, we first propose a two-layer bandit structure for auto tuning the exploration parameter and further generalize it to the Syndicated Bandits framework which can learn multiple hyper-parameters dynamically in contextual bandit environment.
When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations
Vision Transformers (ViTs) and MLPs signal further efforts on replacing hand-wired features or inductive biases with general-purpose neural architectures.
Ranked #5 on
Domain Generalization
on ImageNet-C
(Top 1 Accuracy metric)
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
Based on this observation, we propose a dynamic token sparsification framework to prune redundant tokens progressively and dynamically based on the input.
Ranked #305 on
Image Classification
on ImageNet
Concurrent Adversarial Learning for Large-Batch Training
Current methods usually use extensive data augmentation to increase the batch size, but we found the performance gain with data augmentation decreases as batch size increases, and data augmentation will become insufficient after certain point.
Balancing Robustness and Sensitivity using Feature Contrastive Learning
It is generally believed that robust training of extremely large networks is critical to their success in real-world applications.
Detecting Adversarial Examples with Bayesian Neural Network
In this paper, we propose a new framework to detect adversarial examples motivated by the observations that random components can improve the smoothness of predictors and make it easier to simulate output distribution of deep neural network.
Deep Image Destruction: Vulnerability of Deep Image-to-Image Models against Adversarial Attacks
Recently, the vulnerability of deep image classification models to adversarial attacks has been investigated.
2.5D Visual Relationship Detection
To enable progress on this task, we create a new dataset consisting of 220k human-annotated 2. 5D relationships among 512K objects from 11K images.
On the Sensitivity and Stability of Model Interpretations in NLP
We propose two new criteria, sensitivity and stability, that provide complementary notions of faithfulness to the existed removal-based criteria.
Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation
Our method is able to reveal the hidden model biases not directly shown in the test dataset.
Fast Certified Robust Training with Short Warmup
Despite that state-of-the-art (SOTA) methods including interval bound propagation (IBP) and CROWN-IBP have per-batch training complexity similar to standard neural network training, they usually use a long warmup schedule with hundreds or thousands epochs to reach SOTA performance and are thus still costly.
On the Adversarial Robustness of Vision Transformers
Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision.
Robust and Accurate Object Detection via Adversarial Learning
Data augmentation has become a de facto component for training high-performance deep image classifiers, but its potential is under-explored for object detection.
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verification
Compared to the typically tightest but very costly semidefinite programming (SDP) based incomplete verifiers, we obtain higher verified accuracy with three orders of magnitudes less verification time.
Local Critic Training for Model-Parallel Learning of Deep Neural Networks
We show that the proposed approach successfully decouples the update process of the layer groups for both convolutional neural networks (CNNs) and recurrent neural networks (RNNs).
Robust Reinforcement Learning on State Observations with Learned Optimal Adversary
We study the robustness of reinforcement learning (RL) with adversarially perturbed state observations, which aligns with the setting of many adversarial attacks to deep reinforcement learning (DRL) and is also important for rolling out real-world RL agent under unpredictable sensing noise.
Emotional EEG Classification using Connectivity Features and Convolutional Neural Networks
Convolutional neural networks (CNNs) are widely used to recognize the user's state through electroencephalography (EEG) signals.
Robust Text CAPTCHAs Using Adversarial Examples
At the second stage, we design and apply a highly transferable adversarial attack for text CAPTCHAs to better obstruct CAPTCHA solvers.
Adversarial Masking: Towards Understanding Robustness Trade-off for Generalization
By incorporating different feature maps after the masking, we can distill better features to help model generalization.
Data-aware Low-Rank Compression for Large NLP Models
In this paper, we observe that the learned representation of each layer lies in a low-dimensional space.
Learning to Learn with Smooth Regularization
Recent decades have witnessed great prosperity of deep learning in tackling various problems such as classification and decision making.
Towards Robustness of Deep Neural Networks via Regularization
Recent studies have demonstrated the vulnerability of deep neural networks against adversarial examples.
Learning Contextual Perturbation Budgets for Training Robust Neural Networks
We also demonstrate that the perturbation budget generator can produce semantically-meaningful budgets, which implies that the generator can capture contextual information and the sensitivity of different features in a given image.
Self-Progressing Robust Training
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems.
Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search
This implies that a significant amount of resources can be saved if we are able to stop the searching earlier when we are confident with the current searching result.
Voting based ensemble improves robustness of defensive models
Several previous attempts tackled this problem by ensembling the soft-label prediction and have been proved vulnerable based on the latest attack methods.
Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers
Formal verification of neural networks (NNs) is a challenging and important problem.
On the Transferability of Adversarial Attacksagainst Neural Text Classifier
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
An Efficient Adversarial Attack for Tree Ensembles
We study the problem of efficient adversarial attacks on tree based ensembles such as gradient boosting decision trees (GBDTs) and random forests (RFs).
How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers
For end-to-end efficiency, unlike previous work that assumes random hyperparameter tuning, which over-emphasizes the tuning time, we propose to evaluate with a bandit hyperparameter tuning strategy.
MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation
Knowledge Distillation (KD) has been one of the most popu-lar methods to learn a compact model.
On $\ell_p$-norm Robustness of Ensemble Stumps and Trees
In this paper, we study the problem of robustness verification and certified defense with respect to general $\ell_p$ norm perturbations for ensemble decision stumps and trees.
Improving the Speed and Quality of GAN by Adversarial Training
Generative adversarial networks (GAN) have shown remarkable results in image generation tasks.
Multi-Stage Influence Function
With this score, we can identify the pretraining examples in the pretraining task that contribute most to a prediction in the finetuning task.
Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples
Despite achieving prominent performance on many important tasks, it has been reported that neural networks are vulnerable to adversarial examples.
What Does BERT with Vision Look At?
Pre-trained visually grounded language models such as ViLBERT, LXMERT, and UNITER have achieved significant performance improvement on vision-and-language tasks but what they learn during pre-training remains unclear.
Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples.
DrNAS: Dirichlet Neural Architecture Search
This paper proposes a novel differentiable architecture search method by formulating it into a distribution learning problem.
The Limit of the Batch Size
For the first time we scale the batch size on ImageNet to at least a magnitude larger than all previous work, and provide detailed studies on the performance of many state-of-the-art optimization schemes under this setting.
Provably Robust Metric Learning
Metric learning is an important family of algorithms for classification and similarity search, but the robustness of learned metrics against small adversarial perturbations is less studied.
An Efficient Algorithm For Generalized Linear Bandit: Online Stochastic Gradient Descent and Thompson Sampling
A natural way to resolve this problem is to apply online stochastic gradient descent (SGD) so that the per-step time and memory complexity can be reduced to constant with respect to $t$, but a contextual bandit policy based on online SGD updates that balances exploration and exploitation has remained elusive.
Evaluations and Methods for Explanation through Robustness Analysis
In this paper, we establish a novel set of evaluation criteria for such feature based explanations by robustness analysis.
Spanning Attack: Reinforce Black-box Attacks with Unlabeled Data
By constraining adversarial perturbations in a low-dimensional subspace via spanning an auxiliary unlabeled dataset, the spanning attack significantly improves the query efficiency of a wide variety of existing black-box attacks.
Improved Adversarial Training via Learned Optimizer
Adversarial attack has recently become a tremendous threat to deep learning models.
Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
Several works have shown this vulnerability via adversarial attacks, but existing approaches on improving the robustness of DRL under this setting have limited success and lack for theoretical principles.
Learning to Encode Position for Transformer with Continuous Dynamical Model
The main reason is that position information among input units is not inherently encoded, i. e., the models are permutation equivalent; this problem justifies why all of the existing models are accompanied by a sinusoidal encoding/embedding layer at the input.
Ranked #5 on
Semantic Textual Similarity
on MRPC
Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond
Linear relaxation based perturbation analysis (LiRPA) for neural networks, which computes provable linear bounds of output neurons given a certain amount of input perturbation, has become a core component in robustness verification and certified defense.
CAT: Customized Adversarial Training for Improved Robustness
Adversarial training has become one of the most effective methods for improving robustness of neural networks.
Robustness Verification for Transformers
Robustness verification that aims to formally certify the prediction behavior of neural networks has become an important tool for understanding model behavior and obtaining safety guarantees.
Multiscale Non-stationary Stochastic Bandits
Classic contextual bandit algorithms for linear models, such as LinUCB, assume that the reward distribution for an arm is modeled by a stationary linear regression.
Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
Furthermore, we mathematically show that SDARTS implicitly regularizes the Hessian norm of the validation loss, which accounts for a smoother loss landscape and improved performance.
MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Adversarial training is one of the most popular ways to learn robust models but is usually attack-dependent and time costly.
Overcoming Catastrophic Forgetting by Generative Regularization
In this paper, we propose a new method to overcome catastrophic forgetting by adding generative regularization to Bayesian inference framework.
GraphDefense: Towards Robust Graph Convolutional Networks
Inspired by the previous works on adversarial defense for deep neural networks, and especially adversarial training algorithm, we propose a method called GraphDefense to defend against the adversarial perturbations.
MulCode: A Multiplicative Multi-way Model for Compressing Neural Language Model
For example, input embedding and Softmax matrices in IWSLT-2014 German-to-English data set account for more than 80{\%} of the total model parameters.
Enhancing Certifiable Robustness via a Deep Model Ensemble
We propose an algorithm to enhance certified robustness of a deep model ensemble by optimally weighting each base model.
A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning
In this paper, we proposed a general framework for data poisoning attacks to graph-based semi-supervised learning (G-SSL).
Learning to Learn by Zeroth-Order Oracle
In the learning to learn (L2L) framework, we cast the design of optimization algorithms as a machine learning problem and use deep neural networks to learn the update rules.
Elastic-InfoGAN: Unsupervised Disentangled Representation Learning in Class-Imbalanced Data
We propose a novel unsupervised generative model that learns to disentangle object identity from other low-level aspects in class-imbalanced data.
SSE-PT: Sequential Recommendation Via Personalized Transformer
Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed for better use of the temporal ordering of items that each user has engaged with.
Ranked #1 on
Recommendation Systems
on MovieLens 1M
(nDCG@10 metric)
Stochastically Controlled Compositional Gradient for the Composition problem
The strategy is also accompanied by a mini-batch version of the proposed method that improves query complexity with respect to the size of the mini-batch.
SPROUT: Self-Progressing Robust Training
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy and reliable machine learning systems.
LEARNING TO LEARN WITH BETTER CONVERGENCE
We consider the learning to learn problem, where the goal is to leverage deeplearning models to automatically learn (iterative) optimization algorithms for training machine learning models.
Defending Against Adversarial Examples by Regularized Deep Embedding
Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples.
Sign-OPT: A Query-Efficient Hard-label Adversarial Attack
We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input.
BOSH: An Efficient Meta Algorithm for Decision-based Attacks
Adversarial example generation becomes a viable method for evaluating the robustness of a machine learning model.
Natural Adversarial Sentence Generation with Gradient-based Perturbation
This work proposes a novel algorithm to generate natural language adversarial input for text classification models, in order to investigate the robustness of these models.
Temporal Collaborative Ranking Via Personalized Transformer
Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed us to make better use of the temporal ordering of items that each user has engaged with.
VisualBERT: A Simple and Performant Baseline for Vision and Language
We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks.
Ranked #1 on
Visual Reasoning
on NLVR
On the Robustness of Self-Attentive Models
This work examines the robustness of self-attentive neural networks against adversarial input perturbations.
Efficient Neural Interaction Function Search for Collaborative Filtering
Motivated by the recent success of automated machine learning (AutoML), we propose in this paper the search for simple neural interaction functions (SIF) in CF.
Convergence of Adversarial Training in Overparametrized Neural Networks
Neural networks are vulnerable to adversarial examples, i. e. inputs that are imperceptibly perturbed from natural data and yet incorrectly classified by the network.
Towards Stable and Efficient Training of Verifiably Robust Neural Networks
In this paper, we propose a new certified adversarial training method, CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and a tight linear relaxation based bound, CROWN, in a backward bounding pass.
Robustness Verification of Tree-based Models
We show that there is a simple linear time algorithm for verifying a single tree, and for tree ensembles, the verification problem can be cast as a max-clique problem on a multi-partite graph with bounded boxicity.
Evaluating the Robustness of Nearest Neighbor Classifiers: A Primal-Dual Perspective
Furthermore, we show that dual solutions for these QP problems could give us a valid lower bound of the adversarial perturbation that can be used for formal robustness verification, giving us a nice view of attack/verification for NN models.
ML-LOO: Detecting Adversarial Examples with Feature Attribution
Furthermore, we extend our method to include multi-layer feature attributions in order to tackle the attacks with mixed confidence levels.
Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise
In this paper, we propose a new continuous neural network framework called Neural Stochastic Differential Equation (Neural SDE) network, which naturally incorporates various commonly used regularization mechanisms based on random noise injection.
Evaluating and Enhancing the Robustness of Dialogue Systems: A Case Study on a Negotiation Agent
Moreover, we show that with the adversarial training, we are able to improve the robustness of negotiation agents by 1. 5 points on average against all our attacks.
Graph DNA: Deep Neighborhood Aware Graph Encoding for Collaborative Filtering
In this paper, we propose using Graph DNA, a novel Deep Neighborhood Aware graph encoding algorithm, for exploiting deeper neighborhood information.
Stochastic Shared Embeddings: Data-driven Regularization of Embedding Layers
We find that when used along with widely-used regularization methods such as weight decay and dropout, our proposed SSE can further reduce overfitting, which often leads to more favorable generalization results.
Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
Furthermore, Cluster-GCN allows us to train much deeper GCN without much time and memory overhead, which leads to improved prediction accuracy---using a 5-layer Cluster-GCN, we achieve state-of-the-art test F1 score 99. 36 on the PPI dataset, while the previous best result was 98. 71 by [16].
Ranked #1 on
Node Classification
on Amazon2M
Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach
We study the problem of attacking machine learning models in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions.
Evaluating Robustness of Deep Image Super-Resolution against Adversarial Attacks
Single-image super-resolution aims to generate a high-resolution version of a low-resolution image, which serves as an essential component in many computer vision applications.
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
In this paper, we first study a principled layerwise adaptation strategy to accelerate training of deep neural networks using large mini-batches.
Ranked #11 on
Question Answering
on SQuAD1.1 dev
(F1 metric)
Efficient Contextual Representation Learning With Continuous Outputs
Contextual representation models have achieved great success in improving various downstream natural language processing tasks.
Efficient Contextual Representation Learning Without Softmax Layer
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.
Robust Decision Trees Against Adversarial Examples
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited.
A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
This framework works for neural networks with diverse architectures and nonlinearities and covers both primal and dual views of robustness verification.
Large-Batch Training for LSTM and Beyond
LEGW enables Sqrt Scaling scheme to be useful in practice and as a result we achieve much better results than the Linear Scaling learning rate scheme.
The Limitations of Adversarial Training and the Blind-Spot Attack
In our paper, we shed some lights on the practicality and the hardness of adversarial training by showing that the effectiveness (robustness on test set) of adversarial training has a strong correlation with the distance between a test point and the manifold of training data embedded by the network.
Learning from Group Comparisons: Exploiting Higher Order Interactions
We study the problem of learning from group comparisons, with applications in predicting outcomes of sports and online games.
Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding
Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples.
Block-wise Partitioning for Extreme Multi-label Classification
Extreme multi-label classification aims to learn a classifier that annotates an instance with a relevant subset of labels from an extremely large label set.
Efficient Neural Network Robustness Certification with General Activation Functions
Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem.
Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks
The algorithm achieves an order of magnitude faster inference than the original softmax layer for predicting top-$k$ words in various tasks such as beam search in machine translation or next words prediction.
RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applications
The Jacobian matrix (or the gradient for single-output networks) is directly related to many important properties of neural networks, such as the function landscape, stationary points, (local) Lipschitz constants and robustness to adversarial attacks.
Attack Graph Convolutional Networks by Adding Fake Nodes
In this paper, we propose a new type of "fake node attacks" to attack GCNs by adding malicious fake nodes.
On Extensions of CLEVER: A Neural Network Robustness Evaluation Algorithm
We apply extreme value theory on the new formal robustness guarantee and the estimated robustness is called second-order CLEVER score.
Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way.
Stochastic Second-order Methods for Non-convex Optimization with Inexact Hessian and Gradient
Trust region and cubic regularization methods have demonstrated good performance in small scale non-convex optimization, showing the ability to escape from saddle points.
Stochastically Controlled Stochastic Gradient for the Convex and Non-convex Composition problem
In this paper, we consider the convex and non-convex composition problem with the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where $G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )} $ is the inner function, and $F_i(\cdot)$ is the outer function.
RedSync : Reducing Synchronization Traffic for Distributed Deep Learning
Data parallelism has become a dominant method to scale Deep Neural Network (DNN) training across multiple nodes.
Fast Variance Reduction Method with Stochastic Batch Size
In this paper we study a family of variance reduction methods with randomized batch size---at each step, the algorithm first randomly chooses the batch size and then selects a batch of samples to conduct a variance-reduced stochastic update.
Rob-GAN: Generator, Discriminator, and Adversarial Attacker
Adversarial training is the technique used to improve the robustness of discriminator by combining adversarial attacker and discriminator in the training phase.
Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach
We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions.
Extreme Learning to Rank via Low Rank Assumption
We consider the setting where we wish to perform ranking for hundreds of thousands of users which is common in recommender systems and web search ranking.
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
Model compression is essential for serving large deep neural nets on devices with limited resources or applications that require real-time responses.
Learning Word Embeddings for Low-Resource Languages by PU Learning
In such a situation, the co-occurrence matrix is sparse as the co-occurrences of many word pairs are unobserved.
Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data
We present a probabilistic framework for studying adversarial attacks on discrete data.
Stochastic Zeroth-order Optimization via Variance Reduction method
However, due to the variance in the search direction, the convergence rates and query complexities of existing methods suffer from a factor of $d$, where $d$ is the problem dimension.
AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting.
GenAttack: Practical Black-box Attacks with Gradient-Free Optimization
Our experiments on different datasets (MNIST, CIFAR-10, and ImageNet) show that GenAttack can successfully generate visually imperceptible adversarial examples against state-of-the-art image recognition models with orders of magnitude fewer queries than previous approaches.
LearningWord Embeddings for Low-resource Languages by PU Learning
In such a situation, the co-occurrence matrix is sparse as the co-occurrences of many word pairs are unobserved.
Towards Fast Computation of Certified Robustness for ReLU Networks
Verifying the robustness property of a general Rectified Linear Unit (ReLU) network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer CAV17].
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
In this paper, we study the much more challenging problem of crafting adversarial examples for sequence-to-sequence (seq2seq) models, whose inputs are discrete text strings and outputs have an almost infinite number of possibilities.
SQL-Rank: A Listwise Approach to Collaborative Ranking
In this paper, we propose a listwise approach for constructing user-specific rankings in recommendation systems in a collaborative fashion.
History PCA: A New Algorithm for Streaming PCA
In this paper we propose a new algorithm for streaming principal component analysis.
Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
Our analysis yields a novel robustness metric called CLEVER, which is short for Cross Lipschitz Extreme Value for nEtwork Robustness.
Better Generalization by Efficient Trust Region Method
Solving this subproblem is non-trivial---existing methods have only sub-linear convergence rate.
A comparison of second-order methods for deep convolutional neural networks
Despite many second-order methods have been proposed to train neural networks, most of the results were done on smaller single layer fully connected networks, so we still cannot conclude whether it's useful in training deep convolutional networks.
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning
Our extensive experiments show that our algorithm can successfully craft visually-similar adversarial examples with randomly targeted captions or keywords, and the adversarial examples can be made highly transferable to other image captioning systems.
Towards Robust Neural Networks via Random Self-ensemble
In this paper, we propose a new defense algorithm called Random Self-Ensemble (RSE) by combining two important concepts: {\bf randomness} and {\bf ensemble}.
ImageNet Training in Minutes
If we can make full use of the supercomputer for DNN training, we should be able to finish the 90-epoch ResNet-50 training in one minute.
