Search Results for author: Pin-Yu Chen

Found 264 papers, 117 papers with code

Data-Driven Lipschitz Continuity: A Cost-Effective Approach to Improve Adversarial Robustness

no code implementations28 Jun 2024 Erh-Chung Chen, Pin-Yu Chen, I-Hsin Chung, Che-Rung Lee

This paper aims to provide both a theoretical foundation and a practical solution to ensure the reliability of DNNs.

Adversarial Robustness

Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysis

no code implementations24 Jun 2024 Hongkang Li, Meng Wang, Shuai Zhang, Sijia Liu, Pin-Yu Chen

Efficient training and inference algorithms, such as low-rank adaption and model pruning, have shown impressive performance for learning Transformer-based large foundation models.

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models

1 code implementation14 Jun 2024 Yan Liu, Yu Liu, Xiaokang Chen, Pin-Yu Chen, Daoguang Zan, Min-Yen Kan, Tsung-Yi Ho

As a result, previous debiasing methods mainly finetune or even pre-train language models on newly constructed anti-stereotypical datasets, which are high-cost.

Fairness Language Modelling

PSBD: Prediction Shift Uncertainty Unlocks Backdoor Detection

1 code implementation9 Jun 2024 Wei Li, Pin-Yu Chen, Sijia Liu, Ren Wang

PSBD is motivated by an intriguing Prediction Shift (PS) phenomenon, where poisoned models' predictions on clean data often shift away from true labels towards certain other labels with dropout applied during inference, while backdoor samples exhibit less PS.

A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques

no code implementations7 Jun 2024 Megh Thakkar, Quentin Fournier, Matthew D Riemer, Pin-Yu Chen, Amal Zouaq, Payel Das, Sarath Chandar

Large language models are first pre-trained on trillions of tokens and then instruction-tuned or aligned to specific preferences.

What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding

no code implementations4 Jun 2024 Hongkang Li, Meng Wang, Tengfei Ma, Sijia Liu, Zaixi Zhang, Pin-Yu Chen

Focusing on a graph data model with discriminative nodes that determine node labels and non-discriminative nodes that are class-irrelevant, we characterize the sample complexity required to achieve a desirable generalization error by training with stochastic gradient descent (SGD).

Graph Learning Node Classification

RIGID: A Training-free and Model-Agnostic Framework for Robust AI-Generated Image Detection

no code implementations30 May 2024 Zhiyuan He, Pin-Yu Chen, Tsung-Yi Ho

In particular, the average performance of RIGID exceeds the current best training-free method by more than 25%.

Image Generation

AI Risk Management Should Incorporate Both Safety and Security

no code implementations29 May 2024 Xiangyu Qi, Yangsibo Huang, Yi Zeng, Edoardo Debenedetti, Jonas Geiping, Luxi He, Kaixuan Huang, Udari Madhushani, Vikash Sehwag, Weijia Shi, Boyi Wei, Tinghao Xie, Danqi Chen, Pin-Yu Chen, Jeffrey Ding, Ruoxi Jia, Jiaqi Ma, Arvind Narayanan, Weijie J Su, Mengdi Wang, Chaowei Xiao, Bo Li, Dawn Song, Peter Henderson, Prateek Mittal

The exposure of security vulnerabilities in safety-aligned language models, e. g., susceptibility to adversarial attacks, has shed light on the intricate interplay between AI safety and AI security.

Management

Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models

no code implementations27 May 2024 Chia-Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin-Yu Chen, Chia-Mu Yu, Chun-Ying Huang

Therefore, parameter-efficient fine-tuning such as LoRA have emerged, allowing users to fine-tune LLMs without the need for considerable computing resources, with little performance degradation compared to fine-tuning all parameters.

Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models

no code implementations27 May 2024 Shengyun Peng, Pin-Yu Chen, Matthew Hull, Duen Horng Chau

Safety alignment is the key to guiding the behaviors of large language models (LLMs) that are in line with human preferences and restrict harmful behaviors at inference time, but recent studies show that it can be easily compromised by finetuning with only a few adversarially designed training examples.

A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts

no code implementations26 May 2024 Mohammed Nowaz Rabbani Chowdhury, Meng Wang, Kaoutar El Maghraoui, Naigang Wang, Pin-Yu Chen, Christopher Carothers

The sparsely gated mixture of experts (MoE) architecture sends different inputs to different subnetworks, i. e., experts, through trainable routers.

Binary Classification

SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning

no code implementations24 May 2024 Shuai Zhang, Heshan Devaka Fernando, Miao Liu, Keerthiram Murugesan, Songtao Lu, Pin-Yu Chen, Tianyi Chen, Meng Wang

This paper studies the transfer reinforcement learning (RL) problem where multiple RL problems have different reward functions but share the same underlying transition dynamics.

Q-Learning Reinforcement Learning (RL) +1

Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models

1 code implementation23 May 2024 Yuchen Hu, Chen Chen, Chao-Han Huck Yang, Chengwei Qin, Pin-Yu Chen, Eng Siong Chng, Chao Zhang

We propose an unsupervised adaptation framework, Self-TAught Recognizer (STAR), which leverages unlabeled data to enhance the robustness of automatic speech recognition (ASR) systems in diverse target domains, such as noise and accents.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Improving Transformers using Faithful Positional Encoding

no code implementations15 May 2024 Tsuyoshi Idé, Jokin Labaien, Pin-Yu Chen

We propose a new positional encoding method for a neural network architecture called the Transformer.

Time Series Time Series Classification

Graph is all you need? Lightweight data-agnostic neural architecture search without training

no code implementations2 May 2024 Zhenhan Huang, Tejaswini Pedapati, Pin-Yu Chen, Chunhen Jiang, Jianxi Gao

Our method, dubbed nasgraph, remarkably reduces the computational costs by converting neural architectures to graphs and using the average degree, a graph measure, as the proxy in lieu of the evaluation metric.

Neural Architecture Search

Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks

no code implementations24 Apr 2024 Erh-Chung Chen, Pin-Yu Chen, I-Hsin Chung, Che-Rung Lee

Latency attacks against object detection represent a variant of adversarial attacks that aim to inflate the inference time by generating additional ghost objects in a target image.

object-detection Object Detection

NaNa and MiGu: Semantic Data Augmentation Techniques to Enhance Protein Classification in Graph Neural Networks

1 code implementation21 Mar 2024 Yi-Shan Lan, Pin-Yu Chen, Tsung-Yi Ho

In this paper, we propose novel semantic data augmentation methods, Novel Augmentation of New Node Attributes (NaNa), and Molecular Interactions and Geometric Upgrading (MiGu) to incorporate backbone chemical and side-chain biophysical information into protein classification tasks and a co-embedding residual learning framework.

Data Augmentation Drug Discovery

Larimar: Large Language Models with Episodic Memory Control

no code implementations18 Mar 2024 Payel Das, Subhajit Chaudhury, Elliot Nelson, Igor Melnyk, Sarath Swaminathan, Sihui Dai, Aurélie Lozano, Georgios Kollias, Vijil Chenthamarakshan, Jiří, Navrátil, Soham Dan, Pin-Yu Chen

Efficient and accurate updating of knowledge stored in Large Language Models (LLMs) is one of the most pressing research challenges today.

Model Reprogramming Outperforms Fine-tuning on Out-of-distribution Data in Text-Image Encoders

1 code implementation16 Mar 2024 Andrew Geng, Pin-Yu Chen

When evaluating the performance of a pre-trained model transferred to a downstream task, it is imperative to assess not only the in-distribution (ID) accuracy of the downstream model but also its capacity to generalize and identify out-of-distribution (OOD) samples.

MENTOR: Multilingual tExt detectioN TOward leaRning by analogy

no code implementations12 Mar 2024 Hsin-Ju Lin, Tsu-Chun Chung, Ching-Chun Hsiao, Pin-Yu Chen, Wei-Chen Chiu, Ching-Chun Huang

Text detection is frequently used in vision-based mobile robots when they need to interpret texts in their surroundings to perform a given task.

Few-Shot Learning Scene Text Detection +2

Duwak: Dual Watermarks in Large Language Models

no code implementations12 Mar 2024 Chaoyi Zhu, Jeroen Galjaard, Pin-Yu Chen, Lydia Y. Chen

As large language models (LLM) are increasingly used for text generation tasks, it is critical to audit their usages, govern their applications, and mitigate their potential harms.

Text Generation

Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes

no code implementations1 Mar 2024 Xiaomeng Hu, Pin-Yu Chen, Tsung-Yi Ho

Large Language Models (LLMs) are becoming a prominent generative AI tool, where the user enters a query and the LLM generates an answer.

DiffuseKronA: A Parameter Efficient Fine-tuning Method for Personalized Diffusion Models

no code implementations27 Feb 2024 Shyam Marjit, Harshit Singh, Nityanand Mathur, Sayak Paul, Chia-Mu Yu, Pin-Yu Chen

In the realm of subject-driven text-to-image (T2I) generative models, recent developments like DreamBooth and BLIP-Diffusion have led to impressive results yet encounter limitations due to their intensive fine-tuning demands and substantial parameter requirements.

Image Generation

A Quantum-Classical Collaborative Training Architecture Based on Quantum State Fidelity

no code implementations23 Feb 2024 Ryan L'Abbate, Anthony D'Onofrio Jr., Samuel Stein, Samuel Yen-Chi Chen, Ang Li, Pin-Yu Chen, Juntao Chen, Ying Mao

In this study, we concentrate on quantum deep learning and introduce a collaborative classical-quantum architecture called co-TenQu.

How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?

no code implementations23 Feb 2024 Hongkang Li, Meng Wang, Songtao Lu, Xiaodong Cui, Pin-Yu Chen

Despite the empirical success, the mechanics of how to train a Transformer to achieve ICL and the corresponding ICL capacity is mostly elusive due to the technical challenges of analyzing the nonconvex training problems resulting from the nonlinear self-attention and nonlinear activation in Transformers.

Binary Classification In-Context Learning

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark

1 code implementation18 Feb 2024 Yihua Zhang, Pingzhi Li, Junyuan Hong, Jiaxiang Li, Yimeng Zhang, Wenqing Zheng, Pin-Yu Chen, Jason D. Lee, Wotao Yin, Mingyi Hong, Zhangyang Wang, Sijia Liu, Tianlong Chen

In the evolving landscape of natural language processing (NLP), fine-tuning pre-trained Large Language Models (LLMs) with first-order (FO) optimizers like SGD and Adam has become standard.

Benchmarking

It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition

no code implementations8 Feb 2024 Chen Chen, Ruizhe Li, Yuchen Hu, Sabato Marco Siniscalchi, Pin-Yu Chen, EnSiong Chng, Chao-Han Huck Yang

Recent studies have successfully shown that large language models (LLMs) can be successfully used for generative error correction (GER) on top of the automatic speech recognition (ASR) output.

Audio-Visual Speech Recognition Automatic Speech Recognition +3

From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers

1 code implementation2 Feb 2024 Bharat Runwal, Tejaswini Pedapati, Pin-Yu Chen

Building upon this insight, in this work, we propose a novel density loss that encourages higher activation sparsity (equivalently, lower activation density) in the pre-trained models.

Decoder

DDI-CoCo: A Dataset For Understanding The Effect Of Color Contrast In Machine-Assisted Skin Disease Detection

1 code implementation24 Jan 2024 Ming-Chang Chiu, Yingfei Wang, Yen-Ju Kuo, Pin-Yu Chen

We take another angle to investigate color contrast's impact, beyond skin tones, on malignancy detection in skin disease datasets: We hypothesize that in addition to skin tones, the color difference between the lesion area and skin also plays a role in malignancy detection performance of dermatology AI models.

Image Classification

Large Language Models are Efficient Learners of Noise-Robust Speech Recognition

1 code implementation19 Jan 2024 Yuchen Hu, Chen Chen, Chao-Han Huck Yang, Ruizhe Li, Chao Zhang, Pin-Yu Chen, EnSiong Chng

To this end, we propose to extract a language-space noise embedding from the N-best list to represent the noise conditions of source speech, which can promote the denoising process in GER.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +6

Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift

1 code implementation27 Nov 2023 Shengwei An, Sheng-Yen Chou, Kaiyuan Zhang, QiuLing Xu, Guanhong Tao, Guangyu Shen, Siyuan Cheng, Shiqing Ma, Pin-Yu Chen, Tsung-Yi Ho, Xiangyu Zhang

Diffusion models (DM) have become state-of-the-art generative models because of their capability to generate high-quality images from noises without adversarial training.

Conditional Modeling Based Automatic Video Summarization

no code implementations20 Nov 2023 Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hung Chen, Marcel Worring

The aim of video summarization is to shorten videos automatically while retaining the key information necessary to convey the overall story.

Video Summarization

On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $ε$-Greedy Exploration

no code implementations24 Oct 2023 Shuai Zhang, Hongkang Li, Meng Wang, Miao Liu, Pin-Yu Chen, Songtao Lu, Sijia Liu, Keerthiram Murugesan, Subhajit Chaudhury

This paper provides the first theoretical convergence and sample complexity analysis of the practical setting of DQNs with $\epsilon$-greedy policy.

Q-Learning

Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?

1 code implementation16 Oct 2023 Yu-Lin Tsai, Chia-Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia-You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, Chun-Ying Huang

While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored.

AutoVP: An Automated Visual Prompting Framework and Benchmark

1 code implementation12 Oct 2023 Hsi-Ai Tsao, Lei Hsiung, Pin-Yu Chen, Sijia Liu, Tsung-Yi Ho

To bridge this gap, we propose AutoVP, an end-to-end expandable framework for automating VP design choices, along with 12 downstream image-classification tasks that can serve as a holistic VP-performance benchmark.

Image Classification Visual Prompting

Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

1 code implementation5 Oct 2023 Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, Peter Henderson

Optimizing large language models (LLMs) for downstream use cases often involves the customization of pre-trained LLMs through further fine-tuning.

HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models

1 code implementation NeurIPS 2023 Chen Chen, Yuchen Hu, Chao-Han Huck Yang, Sabato Macro Siniscalchi, Pin-Yu Chen, Eng Siong Chng

We make our results publicly accessible for reproducible pipelines with released pre-trained models, thus providing a new evaluation paradigm for ASR error correction with LLMs.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Masking Improves Contrastive Self-Supervised Learning for ConvNets, and Saliency Tells You Where

1 code implementation22 Sep 2023 Zhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin-Yu Chen, Wei-Chen Chiu

While image data starts to enjoy the simple-but-effective self-supervised learning scheme built upon masking and self-reconstruction objective thanks to the introduction of tokenization procedure and vision transformer backbone, convolutional neural networks as another important and widely-adopted architecture for image data, though having contrastive-learning techniques to drive the self-supervised learning, still face the difficulty of leveraging such straightforward and general masking operation to benefit their learning process significantly.

Contrastive Learning Self-Supervised Learning

Exploring the Benefits of Differentially Private Pre-training and Parameter-Efficient Fine-tuning for Table Transformers

1 code implementation12 Sep 2023 Xilong Wang, Chia-Mu Yu, Pin-Yu Chen

For machine learning with tabular data, Table Transformer (TabTransformer) is a state-of-the-art neural network model, while Differential Privacy (DP) is an essential component to ensure data privacy.

Transfer Learning

Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts

1 code implementation12 Sep 2023 Zhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin-Yu Chen, Wei-Chen Chiu

In this work, we propose Prompting4Debugging (P4D) as a debugging and red-teaming tool that automatically finds problematic prompts for diffusion models to test the reliability of a deployed safety mechanism.

Text-to-Image Generation trustable and focussed LLM generated content

Spectral Adversarial MixUp for Few-Shot Unsupervised Domain Adaptation

1 code implementation3 Sep 2023 Jiajin Zhang, Hanqing Chao, Amit Dhurandhar, Pin-Yu Chen, Ali Tajer, Yangyang Xu, Pingkun Yan

To accomplish this challenging task, first, a spectral sensitivity map is introduced to characterize the generalization weaknesses of models in the frequency domain.

Unsupervised Domain Adaptation

Robust Mixture-of-Expert Training for Convolutional Neural Networks

1 code implementation ICCV 2023 Yihua Zhang, Ruisi Cai, Tianlong Chen, Guanhua Zhang, huan zhang, Pin-Yu Chen, Shiyu Chang, Zhangyang Wang, Sijia Liu

Since the lack of robustness has become one of the main hurdles for CNNs, in this paper we ask: How to adversarially robustify a CNN-based MoE model?

Adversarial Robustness

NeuralFuse: Learning to Recover the Accuracy of Access-Limited Neural Network Inference in Low-Voltage Regimes

1 code implementation29 Jun 2023 Hao-Lun Sun, Lei Hsiung, Nandhini Chandramoorthy, Pin-Yu Chen, Tsung-Yi Ho

To address this challenge, we introduce NeuralFuse, a novel add-on module that addresses the accuracy-energy tradeoff in low-voltage regimes by learning input transformations to generate error-resistant data representations.

VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models

1 code implementation NeurIPS 2023 Sheng-Yen Chou, Pin-Yu Chen, Tsung-Yi Ho

This paper presents a unified backdoor attack framework (VillanDiffusion) to expand the current scope of backdoor analysis for DMs.

Backdoor Attack Denoising

Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks

1 code implementation7 Jun 2023 Mohammed Nowaz Rabbani Chowdhury, Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen

In deep learning, mixture-of-experts (MoE) activates one or few experts (sub-networks) on a per-sample or per-token basis, resulting in significant computation reduction.

Diagnostic Spatio-temporal Transformer with Faithful Encoding

no code implementations26 May 2023 Jokin Labaien, Tsuyoshi Idé, Pin-Yu Chen, Ekhi Zugasti, Xabier De Carlos

This paper addresses the task of anomaly diagnosis when the underlying data generation process has a complex spatio-temporal (ST) dependency.

Time Series Time Series Classification

Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression

no code implementations25 May 2023 Yihao Xue, Siddharth Joshi, Eric Gan, Pin-Yu Chen, Baharan Mirzasoleiman

However, supervised CL is prone to collapsing representations of subclasses within a class by not capturing all their features, and unsupervised CL may suppress harder class-relevant features by focusing on learning easy class-irrelevant features; both significantly compromise representation quality.

Contrastive Learning Representation Learning

Virus2Vec: Viral Sequence Classification Using Machine Learning

no code implementations24 Apr 2023 Sarwan Ali, Babatunde Bello, Prakash Chourasia, Ria Thazhe Punathil, Pin-Yu Chen, Imdad Ullah Khan, Murray Patterson

Understanding the host-specificity of different families of viruses sheds light on the origin of, e. g., SARS-CoV-2, rabies, and other such zoonotic pathogens in humans.

Classification Specificity

GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models

no code implementations19 Apr 2023 Zaitang Li, Pin-Yu Chen, Tsung-Yi Ho

Formally, GREAT Score carries the physical meaning of a global statistic capturing a mean certified attack-proof perturbation level over all samples drawn from a generative model.

Adversarial Robustness

Overload: Latency Attacks on Object Detection for Edge Devices

no code implementations CVPR 2024 Erh-Chung Chen, Pin-Yu Chen, I-Hsin Chung, Che-Rung Lee

Nowadays, the deployment of deep learning-based applications is an essential task owing to the increasing demands on intelligent services.

Object object-detection +1

Exploring the Benefits of Visual Prompting in Differential Privacy

1 code implementation ICCV 2023 Yizhe Li, Yu-Lin Tsai, Xuebin Ren, Chia-Mu Yu, Pin-Yu Chen

Visual Prompting (VP) is an emerging and powerful technique that allows sample-efficient adaptation to downstream tasks by engineering a well-trained frozen source model.

Image Classification Transfer Learning +1

Convex Bounds on the Softmax Function with Applications to Robustness Verification

1 code implementation3 Mar 2023 Dennis Wei, Haoze Wu, Min Wu, Pin-Yu Chen, Clark Barrett, Eitan Farchi

The softmax function is a ubiquitous component at the output of neural networks and increasingly in intermediate layers as well.

MultiRobustBench: Benchmarking Robustness Against Multiple Attacks

no code implementations21 Feb 2023 Sihui Dai, Saeed Mahloujifar, Chong Xiang, Vikash Sehwag, Pin-Yu Chen, Prateek Mittal

Using our framework, we present the first leaderboard, MultiRobustBench, for benchmarking multiattack evaluation which captures performance across attack types and attack strengths.

Benchmarking

A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity

no code implementations12 Feb 2023 Hongkang Li, Meng Wang, Sijia Liu, Pin-Yu Chen

Based on a data model characterizing both label-relevant and label-irrelevant tokens, this paper provides the first theoretical analysis of training a shallow ViT, i. e., one self-attention layer followed by a two-layer perceptron, for a classification task.

Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks

no code implementations6 Feb 2023 Shuai Zhang, Meng Wang, Pin-Yu Chen, Sijia Liu, Songtao Lu, Miao Liu

Due to the significant computational challenge of training large-scale graph neural networks (GNNs), various sparse learning techniques have been exploited to reduce memory and storage costs.

Sparse Learning

Certified Interpretability Robustness for Class Activation Mapping

no code implementations26 Jan 2023 Alex Gu, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, Luca Daniel

Interpreting machine learning models is challenging but crucial for ensuring the safety of deep networks in autonomous driving systems.

Autonomous Driving

AI Maintenance: A Robustness Perspective

no code implementations8 Jan 2023 Pin-Yu Chen, Payel Das

With the advancements in machine learning (ML) methods and compute resources, artificial intelligence (AI) empowered systems are becoming a prevailing technology.

Reprogramming Pretrained Language Models for Protein Sequence Representation Learning

no code implementations5 Jan 2023 Ria Vinod, Pin-Yu Chen, Payel Das

To this end, we reprogram an off-the-shelf pre-trained English language transformer and benchmark it on a set of protein physicochemical prediction tasks (secondary structure, stability, homology, stability) as well as on a biomedically relevant set of protein function prediction tasks (antimicrobial, toxicity, antibody affinity).

Dictionary Learning Language Modelling +3

Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization

no code implementations19 Dec 2022 Zichong Li, Pin-Yu Chen, Sijia Liu, Songtao Lu, Yangyang Xu

In this paper, we design and analyze stochastic inexact augmented Lagrangian methods (Stoc-iALM) to solve problems involving a nonconvex composite (i. e. smooth+nonsmooth) objective and nonconvex smooth functional constraints.

Fairness

Better May Not Be Fairer: A Study on Subgroup Discrepancy in Image Classification

1 code implementation ICCV 2023 Ming-Chang Chiu, Pin-Yu Chen, Xuezhe Ma

In this paper, we provide 20, 000 non-trivial human annotations on popular datasets as a first step to bridge gap to studying how natural semantic spurious features affect image classification, as prior works often study datasets mixing low-level features due to limitations in accessing realistic datasets.

Data Augmentation Image Classification

On Human Visual Contrast Sensitivity and Machine Vision Robustness: A Comparative Study

no code implementations16 Dec 2022 Ming-Chang Chiu, Yingfei Wang, Derrick Eui Gyu Kim, Pin-Yu Chen, Xuezhe Ma

It is well established in neuroscience that color vision plays an essential part in the human visual perception system.

Data Augmentation

How to Backdoor Diffusion Models?

1 code implementation CVPR 2023 Sheng-Yen Chou, Pin-Yu Chen, Tsung-Yi Ho

To gain a better understanding of the limitations and potential risks, this paper presents the first study on the robustness of diffusion models against backdoor attacks.

Backdoor Attack Denoising +1

NCTV: Neural Clamping Toolkit and Visualization for Neural Network Calibration

1 code implementation29 Nov 2022 Lei Hsiung, Yung-Chen Tang, Pin-Yu Chen, Tsung-Yi Ho

With the advancement of deep learning technology, neural networks have demonstrated their excellent ability to provide accurate predictions in many tasks.

Understanding and Improving Visual Prompting: A Label-Mapping Perspective

1 code implementation CVPR 2023 Aochuan Chen, Yuguang Yao, Pin-Yu Chen, Yihua Zhang, Sijia Liu

As highlighted below, we show that when reprogramming an ImageNet-pretrained ResNet-18 to 13 target tasks, our method outperforms baselines by a substantial margin, e. g., 7. 9% and 6. 7% accuracy improvements in transfer learning to the target Flowers102 and CIFAR100 datasets.

Transfer Learning Visual Prompting

Inference and Denoise: Causal Inference-based Neural Speech Enhancement

1 code implementation2 Nov 2022 Tsun-An Hsieh, Chao-Han Huck Yang, Pin-Yu Chen, Sabato Marco Siniscalchi, Yu Tsao

This study addresses the speech enhancement (SE) task within the causal inference paradigm by modeling the noise presence as an intervention.

Causal Inference Speech Enhancement

Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogramming

1 code implementation2 Nov 2022 Yun-Ning Hung, Chao-Han Huck Yang, Pin-Yu Chen, Alexander Lerch

In this work, we introduce a novel method for leveraging pre-trained models for low-resource (music) classification based on the concept of Neural Model Reprogramming (NMR).

Classification Genre classification +3

Certified Robustness of Quantum Classifiers against Adversarial Examples through Quantum Noise

no code implementations2 Nov 2022 Jhih-Cing Huang, Yu-Lin Tsai, Chao-Han Huck Yang, Cheng-Fang Su, Chia-Mu Yu, Pin-Yu Chen, Sy-Yen Kuo

Recently, quantum classifiers have been found to be vulnerable to adversarial attacks, in which quantum classifiers are deceived by imperceptible noises, leading to misclassification.

An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization

1 code implementation27 Oct 2022 Elvin Lo, Pin-Yu Chen

Molecule optimization is an important problem in chemical discovery and has been approached using many techniques, including generative modeling, reinforcement learning, genetic algorithms, and much more.

Visual Prompting for Adversarial Robustness

2 code implementations12 Oct 2022 Aochuan Chen, Peter Lorenz, Yuguang Yao, Pin-Yu Chen, Sijia Liu

In this work, we leverage visual prompting (VP) to improve adversarial robustness of a fixed, pre-trained model at testing time.

Adversarial Defense Adversarial Robustness +1

Rethinking Normalization Methods in Federated Learning

no code implementations7 Oct 2022 Zhixu Du, Jingwei Sun, Ang Li, Pin-Yu Chen, Jianyi Zhang, Hai "Helen" Li, Yiran Chen

We also show that layer normalization is a better choice in FL which can mitigate the external covariate shift and improve the performance of the global model.

Federated Learning

SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data

no code implementations6 Oct 2022 Ching-Yun Ko, Pin-Yu Chen, Jeet Mohapatra, Payel Das, Luca Daniel

Given a pretrained model, the representations of data synthesized from the Gaussian mixture are used to compare with our reference to infer the quality.

Benchmarking Representation Learning

Reprogramming Pretrained Language Models for Antibody Sequence Infilling

1 code implementation5 Oct 2022 Igor Melnyk, Vijil Chenthamarakshan, Pin-Yu Chen, Payel Das, Amit Dhurandhar, Inkit Padhi, Devleena Das

Results on antibody design benchmarks show that our model on low-resourced antibody sequence dataset provides highly diverse CDR sequences, up to more than a two-fold increase of diversity over the baselines, without losing structural integrity and naturalness.

Diversity Language Modelling +2

Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration

no code implementations23 Sep 2022 Yung-Chen Tang, Pin-Yu Chen, Tsung-Yi Ho

Neural network calibration is an essential task in deep learning to ensure consistency between the confidence of model prediction and the true correctness likelihood.

Unraveling the Connections between Privacy and Certified Robustness in Federated Learning Against Poisoning Attacks

no code implementations8 Sep 2022 Chulin Xie, Yunhui Long, Pin-Yu Chen, Qinbin Li, Arash Nourian, Sanmi Koyejo, Bo Li

We then provide two robustness certification criteria: certified prediction and certified attack inefficacy for DPFL on both user and instance levels.

Federated Learning

Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning

no code implementations31 Aug 2022 Zhiyuan He, Yijun Yang, Pin-Yu Chen, Qiang Xu, Tsung-Yi Ho

Empowered by the robust relation net built on SSL, we found that BEYOND outperforms baselines in terms of both detection ability and speed.

Relation Self-Supervised Learning

Active Sampling of Multiple Sources for Sequential Estimation

no code implementations10 Aug 2022 Arpan Mukherjee, Ali Tajer, Pin-Yu Chen, Payel Das

Additionally, each process $i\in\{1, \dots, K\}$ has a private parameter $\alpha_i$.

Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM

1 code implementation20 Jul 2022 Chulin Xie, Pin-Yu Chen, Qinbin Li, Arash Nourian, Ce Zhang, Bo Li

To address these challenges, in this paper, we introduce a VFL framework with multiple heads (VIM), which takes the separate contribution of each client into account, and enables an efficient decomposition of the VFL optimization objective to sub-objectives that can be iteratively tackled by the server and the clients on their own.

Denoising Privacy Preserving +1

Benchmarking Machine Learning Robustness in Covid-19 Genome Sequence Classification

1 code implementation18 Jul 2022 Sarwan Ali, Bikram Sahoo, Alexander Zelikovskiy, Pin-Yu Chen, Murray Patterson

The rapid spread of the COVID-19 pandemic has resulted in an unprecedented amount of sequence data of the SARS-CoV-2 genome -- millions of sequences and counting.

Benchmarking BIG-bench Machine Learning +1

CARBEN: Composite Adversarial Robustness Benchmark

1 code implementation16 Jul 2022 Lei Hsiung, Yun-Yun Tsai, Pin-Yu Chen, Tsung-Yi Ho

Prior literature on adversarial attack methods has mainly focused on attacking with and defending against a single threat model, e. g., perturbations bounded in Lp ball.

Adversarial Attack Adversarial Robustness

Generalization Guarantee of Training Graph Convolutional Networks with Graph Topology Sampling

no code implementations7 Jul 2022 Hongkang Li, Meng Wang, Sijia Liu, Pin-Yu Chen, JinJun Xiong

Graph convolutional networks (GCNs) have recently achieved great empirical success in learning graph-structured data.

Node Classification

On Certifying and Improving Generalization to Unseen Domains

1 code implementation24 Jun 2022 Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, Jihun Hamm

This highlights that the performance of DG methods on a few benchmark datasets may not be representative of their performance on unseen domains in the wild.

Domain Generalization

Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness

1 code implementation15 Jun 2022 Tianlong Chen, huan zhang, Zhenyu Zhang, Shiyu Chang, Sijia Liu, Pin-Yu Chen, Zhangyang Wang

Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish.

Distributed Adversarial Training to Robustify Deep Neural Networks at Scale

2 code implementations13 Jun 2022 Gaoyuan Zhang, Songtao Lu, Yihua Zhang, Xiangyi Chen, Pin-Yu Chen, Quanfu Fan, Lee Martie, Lior Horesh, Mingyi Hong, Sijia Liu

Spurred by that, we propose distributed adversarial training (DAT), a large-batch adversarial training framework implemented over multiple machines.

Distributed Optimization

Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning

1 code implementation8 Jun 2022 Momin Abbas, Quan Xiao, Lisha Chen, Pin-Yu Chen, Tianyi Chen

Model-agnostic meta learning (MAML) is currently one of the dominating approaches for few-shot meta-learning.

Meta-Learning

Theoretical Error Performance Analysis for Variational Quantum Circuit Based Functional Regression

1 code implementation8 Jun 2022 Jun Qi, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hsiu Hsieh

In this work, we first put forth an end-to-end quantum neural network, TTN-VQC, which consists of a quantum tensor network based on a tensor-train network (TTN) for dimensionality reduction and a VQC for functional regression.

Dimensionality Reduction regression

Learning Geometrically Disentangled Representations of Protein Folding Simulations

no code implementations20 May 2022 N. Joseph Tatro, Payel Das, Pin-Yu Chen, Vijil Chenthamarakshan, Rongjie Lai

Massive molecular simulations of drug-target proteins have been used as a tool to understand disease mechanism and develop therapeutics.

Protein Folding

A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Predictions

1 code implementation NAACL 2022 Yong Xie, Dakuo Wang, Pin-Yu Chen, JinJun Xiong, Sijia Liu, Sanmi Koyejo

More and more investors and machine learning models rely on social media (e. g., Twitter and Reddit) to gather real-time information and sentiment to predict stock price movements.

Adversarial Attack Combinatorial Optimization +1

Evaluating the Adversarial Robustness for Fourier Neural Operators

no code implementations8 Apr 2022 Abolaji D. Adesoji, Pin-Yu Chen

In recent years, Machine-Learning (ML)-driven approaches have been widely used in scientific discovery domains.

Adversarial Robustness Super-Resolution

Treatment Learning Causal Transformer for Noisy Image Classification

no code implementations29 Mar 2022 Chao-Han Huck Yang, I-Te Danny Hung, Yi-Chieh Liu, Pin-Yu Chen

In this work, we incorporate this binary information of "existence of noise" as treatment into image classification tasks to improve prediction accuracy by jointly estimating their treatment effects.

Benchmarking Classification +3

Towards Creativity Characterization of Generative Models via Group-based Subset Scanning

no code implementations1 Mar 2022 Celia Cintas, Payel Das, Brian Quanz, Girmaw Abebe Tadesse, Skyler Speakman, Pin-Yu Chen

We propose group-based subset scanning to identify, quantify, and characterize creative processes by detecting a subset of anomalous node-activations in the hidden layers of the generative models.

Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning

1 code implementation22 Feb 2022 Pin-Yu Chen

In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models and can even learn general task-agnostic representations for efficient finetuning to downstream tasks.

BIG-bench Machine Learning Transfer Learning

When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing

no code implementations17 Feb 2022 Chao-Han Huck Yang, Jun Qi, Samuel Yen-Chi Chen, Yu Tsao, Pin-Yu Chen

Our experiments on intent classification show that our proposed BERT-QTC model attains competitive experimental results in the Snips and ATIS spoken language datasets.

Decoder intent-classification +5

Holistic Adversarial Robustness of Deep Learning Models

no code implementations15 Feb 2022 Pin-Yu Chen, Sijia Liu

Adversarial robustness studies the worst-case performance of a machine learning model to ensure safety and reliability.

Adversarial Robustness

Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbations

1 code implementation CVPR 2023 Lei Hsiung, Yun-Yun Tsai, Pin-Yu Chen, Tsung-Yi Ho

We then propose generalized adversarial training (GAT) to extend model robustness from $\ell_{p}$-ball to composite semantic perturbations, such as the combination of Hue, Saturation, Brightness, Contrast, and Rotation.

Adversarial Robustness Scheduling

Auto-Transfer: Learning to Route Transferrable Representations

1 code implementation2 Feb 2022 Keerthiram Murugesan, Vijay Sadashivaiah, Ronny Luss, Karthikeyan Shanmugam, Pin-Yu Chen, Amit Dhurandhar

Knowledge transfer between heterogeneous source and target networks and tasks has received a lot of attention in recent times as large amounts of quality labeled data can be difficult to obtain in many applications.

Transfer Learning

How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis

no code implementations21 Jan 2022 Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen, JinJun Xiong

Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited.

Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics

no code implementations11 Jan 2022 Chunheng Jiang, Tejaswini Pedapati, Pin-Yu Chen, Yizhou Sun, Jianxi Gao

To this end, we construct a network mapping $\phi$, converting a neural network $G_A$ to a directed line graph $G_B$ that is defined on those edges in $G_A$.

Model Selection

Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines

no code implementations1 Dec 2021 Jiachen Sun, Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, Dan Hendrycks, Jihun Hamm, Z. Morley Mao

To alleviate this issue, we propose a novel data augmentation scheme, FourierMix, that produces augmentations to improve the spectral coverage of the training data.

Adversarial Robustness Benchmarking +1

Best Arm Identification in Contaminated Stochastic Bandits

no code implementations NeurIPS 2021 Arpan Mukherjee, Ali Tajer, Pin-Yu Chen, Payel Das

Owing to the adversarial contamination of the rewards, each arm's mean is only partially identifiable.

Formalizing Generalization and Adversarial Robustness of Neural Networks to Weight Perturbations

no code implementations NeurIPS 2021 Yu-Lin Tsai, Chia-Yi Hsu, Chia-Mu Yu, Pin-Yu Chen

Studying the sensitivity of weight perturbation in neural networks and its impacts on model performance, including generalization and robustness, is an active research topic due to its implications on a wide range of machine learning tasks such as model compression, generalization gap assessment, and adversarial attacks.

Adversarial Robustness Model Compression

Pessimistic Model Selection for Offline Deep Reinforcement Learning

no code implementations29 Nov 2021 Chao-Han Huck Yang, Zhengling Qi, Yifan Cui, Pin-Yu Chen

Deep Reinforcement Learning (DRL) has demonstrated great potentials in solving sequential decision making problems in many applications.

Decision Making Model Selection +2

Make an Omelette with Breaking Eggs: Zero-Shot Learning for Novel Attribute Synthesis

no code implementations28 Nov 2021 Yu-Hsuan Li, Tzu-Yin Chao, Ching-Chun Huang, Pin-Yu Chen, Wei-Chen Chiu

Basically, given only a small set of detectors that are learned to recognize some manually annotated attributes (i. e., the seen attributes), we aim to synthesize the detectors of novel attributes in a zero-shot learning manner.

Attribute Classification +1

Meta Adversarial Perturbations

no code implementations AAAI Workshop AdvML 2022 Chia-Hung Yuan, Pin-Yu Chen, Chia-Mu Yu

A plethora of attack methods have been proposed to generate adversarial examples, among which the iterative methods have been demonstrated the ability to find a strong attack.

CAFE: Catastrophic Data Leakage in Vertical Federated Learning

1 code implementation26 Oct 2021 Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, Tianyi Chen

We name our proposed method as catastrophic data leakage in vertical federated learning (CAFE).

Vertical Federated Learning

How and When Adversarial Robustness Transfers in Knowledge Distillation?

no code implementations22 Oct 2021 Rulin Shao, JinFeng Yi, Pin-Yu Chen, Cho-Jui Hsieh

Our comprehensive analysis shows several novel insights that (1) With KDIGA, students can preserve or even exceed the adversarial robustness of the teacher model, even when their models have fundamentally different architectures; (2) KDIGA enables robustness to transfer to pre-trained students, such as KD from an adversarially trained ResNet to a pre-trained ViT, without loss of clean accuracy; and (3) Our derived local linearity bounds for characterizing adversarial robustness in KD are consistent with the empirical results.

Adversarial Robustness Knowledge Distillation +1

Robust Event Classification Using Imperfect Real-world PMU Data

no code implementations19 Oct 2021 Yunchuan Liu, Lei Yang, Amir Ghasemkhani, Hanif Livani, Virgilio A. Centeno, Pin-Yu Chen, Junshan Zhang

Specifically, the data preprocessing step addresses the data quality issues of PMU measurements (e. g., bad data and missing data); in the fine-grained event data extraction step, a model-free event detection method is developed to accurately localize the events from the inaccurate event timestamps in the event logs; and the feature engineering step constructs the event features based on the patterns of different event types, in order to improve the performance and the interpretability of the event classifiers.

Classification Event Detection +1

Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks

no code implementations12 Oct 2021 Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen, JinJun Xiong

Moreover, when the algorithm for training a pruned neural network is specified as an (accelerated) stochastic gradient descent algorithm, we theoretically show that the number of samples required for achieving zero generalization error is proportional to the number of the non-pruned weights in the hidden layer.

Neural Model Reprogramming with Similarity Based Mapping for Low-Resource Spoken Command Recognition

1 code implementation8 Oct 2021 Hao Yen, Pin-Jui Ku, Chao-Han Huck Yang, Hu Hu, Sabato Marco Siniscalchi, Pin-Yu Chen, Yu Tsao

In this study, we propose a novel adversarial reprogramming (AR) approach for low-resource spoken command recognition (SCR), and build an AR-SCR system.

Spoken Command Recognition Transfer Learning

QTN-VQC: An End-to-End Learning framework for Quantum Neural Networks

no code implementations6 Oct 2021 Jun Qi, Chao-Han Huck Yang, Pin-Yu Chen

The advent of noisy intermediate-scale quantum (NISQ) computers raises a crucial challenge to design quantum neural networks for fully quantum learning tasks.

Less is More: Dimension Reduction Finds On-Manifold Adversarial Examples in Hard-Label Attacks

no code implementations29 Sep 2021 Washington Garcia, Pin-Yu Chen, Somesh Jha, Hamilton Scott Clouse, Kevin R. B. Butler

It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.

Dimensionality Reduction Image Classification

Benchmarking Machine Learning Robustness in Covid-19 Spike Sequence Classification

no code implementations29 Sep 2021 Sarwan Ali, Bikram Sahoo, Pin-Yu Chen, Murray Patterson

The rapid spread of the COVID-19 pandemic has resulted in an unprecedented amount of sequence data of the SARS-CoV-2 viral genome --- millions of sequences and counting.

Benchmarking BIG-bench Machine Learning +1

Auto-Transfer: Learning to Route Transferable Representations

no code implementations ICLR 2022 Keerthiram Murugesan, Vijay Sadashivaiah, Ronny Luss, Karthikeyan Shanmugam, Pin-Yu Chen, Amit Dhurandhar

Knowledge transfer between heterogeneous source and target networks and tasks has received a lot of attention in recent times as large amounts of quality labelled data can be difficult to obtain in many applications.

Transfer Learning

How unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis

no code implementations ICLR 2022 Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen, JinJun Xiong

Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited.

Certified Robustness for Free in Differentially Private Federated Learning

no code implementations29 Sep 2021 Chulin Xie, Yunhui Long, Pin-Yu Chen, Krishnaram Kenthapadi, Bo Li

Federated learning (FL) provides an efficient training paradigm to jointly train a global model leveraging data from distributed users.

Federated Learning

Tactics on Refining Decision Boundary for Improving Certification-based Robust Training

no code implementations29 Sep 2021 Wang Zhang, Lam M. Nguyen, Subhro Das, Pin-Yu Chen, Sijia Liu, Alexandre Megretski, Luca Daniel, Tsui-Wei Weng

In verification-based robust training, existing methods utilize relaxation based methods to bound the worst case performance of neural networks given certain perturbation.

Real-World Adversarial Examples involving Makeup Application

no code implementations4 Sep 2021 Chang-Sheng Lin, Chia-Yi Hsu, Pin-Yu Chen, Chia-Mu Yu

The Cycle-GAN is used to generate adversarial makeup, and the architecture of the victimized classifier is VGG 16.

Adversarial Attack Face Recognition +1

Understanding the Limits of Unsupervised Domain Adaptation via Data Poisoning

1 code implementation NeurIPS 2021 Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, Jihun Hamm

Unsupervised domain adaptation (UDA) enables cross-domain learning without target domain labels by transferring knowledge from a labeled source domain whose distribution differs from that of the target.

Data Poisoning Domain Generalization +1

MAML is a Noisy Contrastive Learner in Classification

1 code implementation ICLR 2022 Chia-Hsiang Kao, Wei-Chen Chiu, Pin-Yu Chen

Model-agnostic meta-learning (MAML) is one of the most popular and widely adopted meta-learning algorithms, achieving remarkable success in various learning problems.

Classification Few-Shot Learning

Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design

1 code implementation24 Jun 2021 Yue Cao, Payel Das, Vijil Chenthamarakshan, Pin-Yu Chen, Igor Melnyk, Yang shen

Designing novel protein sequences for a desired 3D topological fold is a fundamental yet non-trivial task in protein engineering.

Protein Design

Generalizing Adversarial Training to Composite Semantic Perturbations

no code implementations ICML Workshop AML 2021 Yun-Yun Tsai, Lei Hsiung, Pin-Yu Chen, Tsung-Yi Ho

We then propose generalized adversarial training (GAT) to extend model robustness from $\ell_{p}$ norm to composite semantic perturbations, such as Hue, Saturation, Brightness, Contrast, and Rotation.

Scheduling

CRFL: Certifiably Robust Federated Learning against Backdoor Attacks

1 code implementation15 Jun 2021 Chulin Xie, Minghao Chen, Pin-Yu Chen, Bo Li

Our method exploits clipping and smoothing on model parameters to control the global model smoothness, which yields a sample-wise robustness certification on backdoors with limited magnitude.

Federated Learning

Predicting Deep Neural Network Generalization with Perturbation Response Curves

no code implementations NeurIPS 2021 Yair Schiff, Brian Quanz, Payel Das, Pin-Yu Chen

However, despite these successes, the recent Predicting Generalization in Deep Learning (PGDL) NeurIPS 2020 competition suggests that there is a need for more robust and efficient measures of network generalization.

Simple Transparent Adversarial Examples

no code implementations20 May 2021 Jaydeep Borkar, Pin-Yu Chen

We propose two new aspects of adversarial image generation methods and evaluate them on the robustness of Google Cloud Vision API's optical character recognition service and object detection APIs deployed in real-world settings such as sightengine. com, picpurify. com, Google Cloud Vision API, and Microsoft Azure's Computer Vision API.

Image Generation object-detection +3

Vision Transformers are Robust Learners

1 code implementation17 May 2021 Sayak Paul, Pin-Yu Chen

Transformers, composed of multiple self-attention layers, hold strong promises toward a generic learning primitive applicable to different data modalities, including the recent breakthroughs in computer vision achieving state-of-the-art (SOTA) standard accuracy.

Anomaly Detection Image Classification +1

High-Robustness, Low-Transferability Fingerprinting of Neural Networks

no code implementations14 May 2021 Siyue Wang, Xiao Wang, Pin-Yu Chen, Pu Zhao, Xue Lin

This paper proposes Characteristic Examples for effectively fingerprinting deep neural networks, featuring high-robustness to the base model against model pruning as well as low-transferability to unassociated models.

Vocal Bursts Intensity Prediction

Gi and Pal Scores: Deep Neural Network Generalization Statistics

no code implementations8 Apr 2021 Yair Schiff, Brian Quanz, Payel Das, Pin-Yu Chen

The field of Deep Learning is rich with empirical evidence of human-like performance on a variety of regression, classification, and control tasks.

regression

Towards creativity characterization of generative models via group-based subset scanning

no code implementations1 Apr 2021 Celia Cintas, Payel Das, Brian Quanz, Skyler Speakman, Victor Akinwande, Pin-Yu Chen

We propose group-based subset scanning to quantify, detect, and characterize creative processes by detecting a subset of anomalous node-activations in the hidden layers of generative models.

On the Adversarial Robustness of Vision Transformers

1 code implementation29 Mar 2021 Rulin Shao, Zhouxing Shi, JinFeng Yi, Pin-Yu Chen, Cho-Jui Hsieh

Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision.

Adversarial Robustness

Don't Forget to Sign the Gradients!

1 code implementation5 Mar 2021 Omid Aramoon, Pin-Yu Chen, Gang Qu

Engineering a top-notch deep learning model is an expensive procedure that involves collecting data, hiring human resources with expertise in machine learning, and providing high computational resources.

Image Classification

Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples

no code implementations4 Mar 2021 Washington Garcia, Pin-Yu Chen, Somesh Jha, Scott Clouse, Kevin R. B. Butler

It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.

Dimensionality Reduction Image Classification

Formalizing Generalization and Robustness of Neural Networks to Weight Perturbations

no code implementations3 Mar 2021 Yu-Lin Tsai, Chia-Yi Hsu, Chia-Mu Yu, Pin-Yu Chen

Studying the sensitivity of weight perturbation in neural networks and its impacts on model performance, including generalization and robustness, is an active research topic due to its implications on a wide range of machine learning tasks such as model compression, generalization gap assessment, and adversarial attacks.

Model Compression

Adversarial Examples can be Effective Data Augmentation for Unsupervised Machine Learning

1 code implementation2 Mar 2021 Chia-Yi Hsu, Pin-Yu Chen, Songtao Lu, Sijia Liu, Chia-Mu Yu

In this paper, we propose a framework of generating adversarial examples for unsupervised models and demonstrate novel applications to data augmentation.

BIG-bench Machine Learning Contrastive Learning +2

Domain Adaptation for Learning Generator from Paired Few-Shot Data

no code implementations25 Feb 2021 Chun-Chih Teng, Pin-Yu Chen, Wei-Chen Chiu

We propose a Paired Few-shot GAN (PFS-GAN) model for learning generators with sufficient source data and a few target data.

Diversity Domain Adaptation +1

Non-Singular Adversarial Robustness of Neural Networks

no code implementations23 Feb 2021 Yu-Lin Tsai, Chia-Yi Hsu, Chia-Mu Yu, Pin-Yu Chen

In this paper, we formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights.

Adversarial Robustness

On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

1 code implementation ICLR 2021 Ren Wang, Kaidi Xu, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Chuang Gan, Meng Wang

Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning.

Adversarial Attack Adversarial Robustness +3

Training a Resilient Q-Network against Observational Interference

1 code implementation18 Feb 2021 Chao-Han Huck Yang, I-Te Danny Hung, Yi Ouyang, Pin-Yu Chen

Deep reinforcement learning (DRL) has demonstrated impressive performance in various gaming simulators and real-world applications.

Causal Inference

Meta Federated Learning

no code implementations10 Feb 2021 Omid Aramoon, Pin-Yu Chen, Gang Qu, Yuan Tian

Due to its distributed methodology alongside its privacy-preserving features, Federated Learning (FL) is vulnerable to training time adversarial attacks.

Federated Learning Privacy Preserving

Fast Training of Provably Robust Neural Networks by SingleProp

no code implementations1 Feb 2021 Akhilan Boopathy, Tsui-Wei Weng, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, Luca Daniel

Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees.

Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning

no code implementations1 Feb 2021 Syed Zawad, Ahsan Ali, Pin-Yu Chen, Ali Anwar, Yi Zhou, Nathalie Baracaldo, Yuan Tian, Feng Yan

Data heterogeneity has been identified as one of the key features in federated learning but often overlooked in the lens of robustness to adversarial attacks.

Federated Learning

Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks

no code implementations30 Jan 2021 Maurício Gruppi, Sibel Adali, Pin-Yu Chen

The goal of LSC is to characterize and quantify language variations with respect to word meaning, to measure how distinct two language sources are (that is, people or language models).

Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records

no code implementations13 Jan 2021 Yiqin Yu, Pin-Yu Chen, Yuan Zhou, Jing Mei

With the successful adoption of machine learning on electronic health records (EHRs), numerous computational models have been deployed to address a variety of clinical problems.

Data Augmentation Domain Adaptation

Robust Text CAPTCHAs Using Adversarial Examples

no code implementations7 Jan 2021 Rulin Shao, Zhouxing Shi, JinFeng Yi, Pin-Yu Chen, Cho-Jui Hsieh

At the second stage, we design and apply a highly transferable adversarial attack for text CAPTCHAs to better obstruct CAPTCHA solvers.

Adversarial Attack Optical Character Recognition (OCR)

Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks

no code implementations NeurIPS 2021 Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen, JinJun Xiong

Moreover, as the algorithm for training a sparse neural network is specified as (accelerated) stochastic gradient descent algorithm, we theoretically show that the number of samples required for achieving zero generalization error is proportional to the number of the non-pruned model weights in the hidden layer.

CAFE: Catastrophic Data Leakage in Federated Learning

no code implementations1 Jan 2021 Xiao Jin, Ruijie Du, Pin-Yu Chen, Tianyi Chen

In this paper, we revisit this defense premise and propose an advanced data leakage attack to efficiently recover batch data from the shared aggregated gradients.

Federated Learning

ProGAE: A Geometric Autoencoder-based Generative Model for Disentangling Protein Dynamics

no code implementations1 Jan 2021 Norman Joseph Tatro, Payel Das, Pin-Yu Chen, Vijil Chenthamarakshan, Rongjie Lai

Empowered by the disentangled latent space learning, the extrinsic latent embedding is successfully used for classification or property prediction of different drugs bound to a specific protein.