Search Results for author:
Found 90 papers, 56 papers with code
On Lp-norm Robustness of Ensemble Decision Stumps and Trees
In this paper, we study the robustness verification and defense with respect to general $\ell_p$ norm perturbation for ensemble trees and stumps.
Using Explainable AI and Transfer Learning to understand and predict the maintenance of Atlantic blocking with limited observational data
This work demonstrates the potential for machine learning methods to extract meaningful precursors of extreme weather events and achieve better prediction using limited observational data.
Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation for Efficient Synthesis and Verification
The flexibility and efficiency of our framework allow us to demonstrate Lyapunov-stable output feedback control with synthesized NN-based controllers and NN-based observers with formal stability guarantees, for the first time in literature.
Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations
Sequential-in-time methods solve a sequence of training problems to fit nonlinear parametrizations such as neural networks to approximate solution trajectories of partial differential equations over time.
WavCraft: Audio Editing and Generation with Natural Language Prompts
We introduce WavCraft, a collective system that leverages large language models (LLMs) to connect diverse task-specific models for audio content creation and editing.
A Safe Screening Rule with Bi-level Optimization of $ν$ Support Vector Machine
Support vector machine (SVM) has achieved many successes in machine learning, especially for a small sample problem.
COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability
Based on this connection, we adapt the Energy-based Constrained Decoding with Langevin Dynamics (COLD), a state-of-the-art, highly efficient algorithm in controllable text generation, and introduce the COLD-Attack framework which unifies and automates the search of adversarial LLM attacks under a variety of control requirements such as fluency, stealthiness, sentiment, and left-right-coherence.
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
Solving Expensive Optimization Problems in Dynamic Environments with Meta-learning
Although data-driven evolutionary optimization and Bayesian optimization (BO) approaches have shown promise in solving expensive optimization problems in static environments, the attempts to develop such approaches in dynamic environments remain rarely unexplored.
HoneyBee: Progressive Instruction Finetuning of Large Language Models for Materials Science
We propose an instruction-based process for trustworthy data curation in materials science (MatSci-Instruct), which we then apply to finetune a LLaMa-based language model targeted for materials science (HoneyBee).
AG-CRC: Anatomy-Guided Colorectal Cancer Segmentation in CT with Imperfect Anatomical Knowledge
When delineating lesions from medical images, a human expert can always keep in mind the anatomical structure behind the voxels.
Symbolic Music Representations for Classification Tasks: A Systematic Evaluation
Music Information Retrieval (MIR) has seen a recent surge in deep learning-based approaches, which often involve encoding symbolic music (i. e., music represented in terms of discrete note events) in an image-like or language like fashion.
DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothing
Diffusion models have been leveraged to perform adversarial purification and thus provide both empirical and certified robustness for a standard model.
Robust Mixture-of-Expert Training for Convolutional Neural Networks
Since the lack of robustness has become one of the main hurdles for CNNs, in this paper we ask: How to adversarially robustify a CNN-based MoE model?
Improving the Generalization Ability in Essay Coherence Evaluation through Monotonic Constraints
To address these concerns, we propose a coherence scoring model consisting of a regression model with two feature extractors: a local coherence discriminative model and a punctuation correction model.
Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectories
With extensive experimental evaluation, we show the prevalence of \emph{generalization failure} on controllable states from stranger agents.
Queer In AI: A Case Study in Community-Led Participatory AI
We present Queer in AI as a case study for community-led participatory design in AI.
Provably Bounding Neural Network Preimages
Most work on the formal verification of neural networks has focused on bounding the set of outputs that correspond to a given set of inputs (for example, bounded perturbations of a nominal input).
Understanding and eliminating spurious modes in variational Monte Carlo using collective variables
The use of neural network parametrizations to represent the ground state in variational Monte Carlo (VMC) calculations has generated intense interest in recent years.
Are AlphaZero-like Agents Robust to Adversarial Perturbations?
Given that the state space of Go is extremely large and a human player can play the game from any legal state, we ask whether adversarial states exist for Go AIs that may lead them to play surprisingly wrong actions.
FI-ODE: Certifiably Robust Forward Invariance in Neural ODEs
Forward invariance is a long-studied property in control theory that is used to certify that a dynamical system stays within some pre-specified set of states for all time, and also admits robustness guarantees (e. g., the certificate holds under perturbations).
Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagation
In this paper, we develop an efficient framework for computing the $\ell_\infty$ local Lipschitz constant of a neural network by tightly upper bounding the norm of Clarke Jacobian via linear bound propagation.
General Cutting Planes for Bound-Propagation-Based Neural Network Verification
Our generalized bound propagation method, GCP-CROWN, opens up the opportunity to apply general cutting plane methods for neural network verification while benefiting from the efficiency and GPU acceleration of bound propagation methods.
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness
Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish.
On the Robustness of Safe Reinforcement Learning under Observational Perturbations
One interesting and counter-intuitive finding is that the maximum reward attack is strong, as it can both induce unsafe behaviors and make the attack stealthy by maintaining the reward.
BronchusNet: Region and Structure Prior Embedded Representation Learning for Bronchus Segmentation and Classification
CT-based bronchial tree analysis plays an important role in the computer-aided diagnosis for respiratory diseases, as it could provide structured information for clinicians.
COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
We leverage COPA to certify three RL environments trained with different algorithms and conclude: (1) The proposed robust aggregation protocols such as temporal aggregation can significantly improve the certifications; (2) Our certification for both per-state action stability and cumulative reward bound are efficient and tight; (3) The certification for different training algorithms and environments are different, implying their intrinsic robustness properties.
Sharpness-Aware Minimization with Dynamic Reweighting
Deep neural networks are often overparameterized and may not easily achieve model generalization.
Temporal Shuffling for Defending Deep Action Recognition Models against Adversarial Attacks
Recently, video-based action recognition methods using convolutional neural networks (CNNs) achieve remarkable recognition performance.
Robustness between the worst and average case
Several recent works in machine learning have focused on evaluating the test-time robustness of a classifier: how well the classifier performs not just on the target domain it was trained upon, but upon perturbed examples.
Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds
Certified robustness is a desirable property for deep neural networks in safety-critical applications, and popular training algorithms can certify robustness of a neural network by computing a global bound on its Lipschitz constant.
Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training
Due to the proliferation of renewable energy and its intrinsic intermittency and stochasticity, current power systems face severe operational challenges.
A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks
In this work, we formulate an adversarial attack using a branch-and-bound (BaB) procedure on ReLU neural networks and search adversarial examples in the activation space corresponding to binary variables in a mixed integer programming (MIP) formulation.
Empirical robustification of pre-trained classifiers
Most pre-trained classifiers, though they may work extremely well on the domain they were trained upon, are not trained in a robust fashion, and therefore are sensitive to adversarial attacks.
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification
We develop $\beta$-CROWN, a new bound propagation based method that can fully encode neuron split constraints in branch-and-bound (BaB) based complete verification via optimizable parameters $\beta$.
Deep Image Destruction: Vulnerability of Deep Image-to-Image Models against Adversarial Attacks
Recently, the vulnerability of deep image classification models to adversarial attacks has been investigated.
Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation
Our method is able to reveal the hidden model biases not directly shown in the test dataset.
Fast Certified Robust Training with Short Warmup
Despite that state-of-the-art (SOTA) methods including interval bound propagation (IBP) and CROWN-IBP have per-batch training complexity similar to standard neural network training, they usually use a long warmup schedule with hundreds or thousands epochs to reach SOTA performance and are thus still costly.
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verification
Compared to the typically tightest but very costly semidefinite programming (SDP) based incomplete verifiers, we obtain higher verified accuracy with three orders of magnitudes less verification time.
Just Noticeable Difference for Deep Machine Vision
Experimental results on image classification demonstrate that we successfully find the JND for deep machine vision.
Robust Reinforcement Learning on State Observations with Learned Optimal Adversary
We study the robustness of reinforcement learning (RL) with adversarially perturbed state observations, which aligns with the setting of many adversarial attacks to deep reinforcement learning (DRL) and is also important for rolling out real-world RL agent under unpredictable sensing noise.
Learning Contextual Perturbation Budgets for Training Robust Neural Networks
We also demonstrate that the perturbation budget generator can produce semantically-meaningful budgets, which implies that the generator can capture contextual information and the sensitivity of different features in a given image.
Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers
Formal verification of neural networks (NNs) is a challenging and important problem.
An Efficient Adversarial Attack for Tree Ensembles
We study the problem of efficient adversarial attacks on tree based ensembles such as gradient boosting decision trees (GBDTs) and random forests (RFs).
On $\ell_p$-norm Robustness of Ensemble Stumps and Trees
In this paper, we study the problem of robustness verification and certified defense with respect to general $\ell_p$ norm perturbations for ensemble decision stumps and trees.
The Limit of the Batch Size
For the first time we scale the batch size on ImageNet to at least a magnitude larger than all previous work, and provide detailed studies on the performance of many state-of-the-art optimization schemes under this setting.
Spanning Attack: Reinforce Black-box Attacks with Unlabeled Data
By constraining adversarial perturbations in a low-dimensional subspace via spanning an auxiliary unlabeled dataset, the spanning attack significantly improves the query efficiency of a wide variety of existing black-box attacks.
Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation
There are plenty of studies showing that the knowledge distillation is efficient in transferring the knowledge from BERT into the model with a smaller size of parameters.
Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
Several works have shown this vulnerability via adversarial attacks, but existing approaches on improving the robustness of DRL under this setting have limited success and lack for theoretical principles.
Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond
Linear relaxation based perturbation analysis (LiRPA) for neural networks, which computes provable linear bounds of output neurons given a certain amount of input perturbation, has become a core component in robustness verification and certified defense.
Robustness Verification for Transformers
Robustness verification that aims to formally certify the prediction behavior of neural networks has become an important tool for understanding model behavior and obtaining safety guarantees.
MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Adversarial training is one of the most popular ways to learn robust models but is usually attack-dependent and time costly.
MFPN: A Novel Mixture Feature Pyramid Network of Multiple Architectures for Object Detection
Feature pyramids are widely exploited in many detectors to solve the scale variation problem for object detection.
Robust Triple-Matrix-Recovery-Based Auto-Weighted Label Propagation for Classification
Our method can jointly re-cover the underlying clean data, clean labels and clean weighting spaces by decomposing the original data, predicted soft labels or weights into a clean part plus an error part by fitting noise.
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
This paper aims to quantify and reduce a particular type of bias exhibited by language models: bias in the sentiment of generated text.
Enhancing Certifiable Robustness via a Deep Model Ensemble
We propose an algorithm to enhance certified robustness of a deep model ensemble by optimally weighting each base model.
LocalGAN: Modeling Local Distributions for Adversarial Response Generation
This paper presents a new methodology for modeling the local semantic distribution of responses to a given query in the human-conversation corpus, and on this basis, explores a specified adversarial learning mechanism for training Neural Response Generation (NRG) models to build conversational agents.
MemeFaceGenerator: Adversarial Synthesis of Chinese Meme-face from Natural Sentences
Chinese meme-face is a special kind of internet subculture widely spread in Chinese Social Community Networks.
Defending Against Adversarial Attacks Using Random Forests
As deep neural networks (DNNs) have become increasingly important and popular, the robustness of DNNs is the key to the safety of both the Internet and the physical world.
Towards Stable and Efficient Training of Verifiably Robust Neural Networks
In this paper, we propose a new certified adversarial training method, CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and a tight linear relaxation based bound, CROWN, in a backward bounding pass.
Robustness Verification of Tree-based Models
We show that there is a simple linear time algorithm for verifying a single tree, and for tree ensembles, the verification problem can be cast as a max-clique problem on a multi-partite graph with bounded boxicity.
Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
In this paper, we employ adversarial training to improve the performance of randomized smoothing.
Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach
We study the problem of attacking machine learning models in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions.
How Training Data Affect the Accuracy and Robustness of Neural Networks for Image Classification
In light of a recent study on the mutual influence between robustness and accuracy over 18 different ImageNet models, this paper investigates how training data affect the accuracy and robustness of deep neural networks.
Minimum Divergence vs. Maximum Margin: an Empirical Comparison on Seq2Seq Models
Sequence to sequence (seq2seq) models have become a popular framework for neural sequence prediction.
Evaluating Robustness of Deep Image Super-Resolution against Adversarial Attacks
Single-image super-resolution aims to generate a high-resolution version of a low-resolution image, which serves as an essential component in many computer vision applications.
Adversarial Robustness vs Model Compression, or Both?
Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting, training a small model from scratch even with inherited initialization from the large model cannot achieve both adversarial robustness and high standard accuracy.
Robust Decision Trees Against Adversarial Examples
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited.
A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
This framework works for neural networks with diverse architectures and nonlinearities and covers both primal and dual views of robustness verification.
The Limitations of Adversarial Training and the Blind-Spot Attack
In our paper, we shed some lights on the practicality and the hardness of adversarial training by showing that the effectiveness (robustness on test set) of adversarial training has a strong correlation with the distance between a test point and the manifold of training data embedded by the network.
Efficient Neural Network Robustness Certification with General Activation Functions
Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem.
RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applications
The Jacobian matrix (or the gradient for single-output networks) is directly related to many important properties of neural networks, such as the function landscape, stationary points, (local) Lipschitz constants and robustness to adversarial attacks.
On Extensions of CLEVER: A Neural Network Robustness Evaluation Algorithm
We apply extreme value theory on the new formal robustness guarantee and the estimated robustness is called second-order CLEVER score.
Structured Adversarial Attack: Towards General Implementation and Better Interpretability
When generating adversarial examples to attack deep neural networks (DNNs), Lp norm of the added perturbation is usually used to measure the similarity between original image and adversarial example.
Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models
The prediction accuracy has been the long-lasting and sole standard for comparing the performance of different image classification models, including the ImageNet competition.
Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach
We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions.
AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting.
GenAttack: Practical Black-box Attacks with Gradient-Free Optimization
Our experiments on different datasets (MNIST, CIFAR-10, and ImageNet) show that GenAttack can successfully generate visually imperceptible adversarial examples against state-of-the-art image recognition models with orders of magnitude fewer queries than previous approaches.
Towards Fast Computation of Certified Robustness for ReLU Networks
Verifying the robustness property of a general Rectified Linear Unit (ReLU) network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer CAV17].
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
In this paper, we study the much more challenging problem of crafting adversarial examples for sequence-to-sequence (seq2seq) models, whose inputs are discrete text strings and outputs have an almost infinite number of possibilities.
Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
Our analysis yields a novel robustness metric called CLEVER, which is short for Cross Lipschitz Extreme Value for nEtwork Robustness.
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning
Our extensive experiments show that our algorithm can successfully craft visually-similar adversarial examples with randomly targeted captions or keywords, and the adversarial examples can be made highly transferable to other image captioning systems.
Towards Robust Neural Networks via Random Self-ensemble
In this paper, we propose a new defense algorithm called Random Self-Ensemble (RSE) by combining two important concepts: {\bf randomness} and {\bf ensemble}.
EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
Recent studies have highlighted the vulnerability of deep neural networks (DNNs) to adversarial examples - a visually indistinguishable adversarial image can easily be crafted to cause a well-trained model to misclassify.
ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
However, different from leveraging attack transferability from substitute models, we propose zeroth order optimization (ZOO) based attacks to directly estimate the gradients of the targeted DNN for generating adversarial examples.
Gradient Boosted Decision Trees for High Dimensional Sparse Output
In this paper, we study the gradient boosted decision trees (GBDT) when the output space is high dimensional and sparse.
GPU-acceleration for Large-scale Tree Boosting
In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training.
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
On network configurations with low bandwidth or high latency, D-PSGD can be up to one order of magnitude faster than its well-optimized centralized counterparts.
Sublinear Time Orthogonal Tensor Decomposition
We show in a number of cases one can achieve the same theoretical guarantees in sublinear time, i. e., even without reading most of the input tensor.
