Search Results for author: Hongyang Zhang

Found 47 papers, 20 papers with code

AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls

1 code implementation6 Feb 2024 Yu Du, Fangyun Wei, Hongyang Zhang

We also revisit the evaluation protocol introduced by previous works and identify a limitation in this protocol that leads to an artificially high pass rate.

Language Modelling Large Language Model

DiPmark: A Stealthy, Efficient and Resilient Watermark for Large Language Models

no code implementations11 Oct 2023 Yihan Wu, Zhengmian Hu, Hongyang Zhang, Heng Huang

Watermarking techniques offer a promising way to secure data via embedding covert information into the data.

Language Modelling

RAIN: Your Language Models Can Align Themselves without Finetuning

1 code implementation13 Sep 2023 Yuhui Li, Fangyun Wei, Jinjing Zhao, Chao Zhang, Hongyang Zhang

We discover that by integrating self-evaluation and rewind mechanisms, unaligned LLMs can directly produce responses consistent with human preferences via self-boosting.

Adversarial Attack

zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training

1 code implementation30 Jul 2023 Haochen Sun, Tonghe Bai, Jason Li, Hongyang Zhang

In response to this challenge, we present zero-knowledge deep learning (zkDL), an efficient zero-knowledge proof for deep learning training.

Gradient-Based Word Substitution for Obstinate Adversarial Examples Generation in Language Models

no code implementations24 Jul 2023 Yimu Wang, Peng Shi, Hongyang Zhang

Furthermore, to show the transferability of obstinate word substitutions found by GradObstinate, we replace the words in four representative NLP benchmarks with their obstinate substitutions.

Memorization MRPC +1

Cooperation or Competition: Avoiding Player Domination for Multi-Target Robustness via Adaptive Budgets

no code implementations CVPR 2023 Yimu Wang, Dinghuai Zhang, Yihan Wu, Heng Huang, Hongyang Zhang

We identify a phenomenon named player domination in the bargaining game, namely that the existing max-based approaches, such as MAX and MSD, do not converge.

Understanding the Impact of Adversarial Robustness on Accuracy Disparity

1 code implementation28 Nov 2022 Yuzheng Hu, Fan Wu, Hongyang Zhang, Han Zhao

More specifically, we demonstrate that while the constraint of adversarial robustness consistently degrades the standard accuracy in the balanced class setting, the class imbalance ratio plays a fundamentally different role in accuracy disparity compared to the Gaussian case, due to the heavy tail of the stable distribution.

Adversarial Robustness Open-Ended Question Answering

Direct-Effect Risk Minimization for Domain Generalization

1 code implementation26 Nov 2022 Yuhui Li, Zejia Wu, Chao Zhang, Hongyang Zhang

In this work, we introduce the concepts of direct and indirect effects from causal inference to the domain generalization problem.

Causal Inference Domain Generalization +1

Towards Robust Dataset Learning

1 code implementation19 Nov 2022 Yihan Wu, Xinda Li, Florian Kerschbaum, Heng Huang, Hongyang Zhang

In this paper, we study the problem of learning a robust dataset such that any classifier naturally trained on the dataset is adversarially robust.

A Closer Look at Robustness to L-infinity and Spatial Perturbations and their Composition

no code implementations5 Oct 2022 Luke Rowe, Benjamin Thérien, Krzysztof Czarnecki, Hongyang Zhang

In adversarial machine learning, the popular $\ell_\infty$ threat model has been the focus of much previous work.

RetrievalGuard: Provably Robust 1-Nearest Neighbor Image Retrieval

no code implementations17 Jun 2022 Yihan Wu, Hongyang Zhang, Heng Huang

The challenge is to design a provably robust algorithm that takes into consideration the 1-NN search and the high-dimensional nature of the embedding space.

Image Retrieval Retrieval

Causal Balancing for Domain Generalization

1 code implementation10 Jun 2022 Xinyi Wang, Michael Saxon, Jiachen Li, Hongyang Zhang, Kun Zhang, William Yang Wang

While machine learning models rapidly advance the state-of-the-art on various real-world tasks, out-of-domain (OOD) generalization remains a challenging problem given the vulnerability of these models to spurious correlations.

Domain Generalization

Building Robust Ensembles via Margin Boosting

1 code implementation7 Jun 2022 Dinghuai Zhang, Hongyang Zhang, Aaron Courville, Yoshua Bengio, Pradeep Ravikumar, Arun Sai Suggala

Consequently, an emerging line of work has focused on learning an ensemble of neural networks to defend against adversarial attacks.

Adversarial Robustness

Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking

1 code implementation19 May 2022 Minghan Li, Xinyu Zhang, Ji Xin, Hongyang Zhang, Jimmy Lin

For example, on MS MARCO Passage v1, our method yields an average candidate set size of 27 out of 1, 000 which increases the reranking speed by about 37 times, while the MRR@10 is greater than a pre-specified value of 0. 38 with about 90% empirical coverage and the empirical baselines fail to provide such guarantee.

Computational Efficiency Information Retrieval +1

A Law of Robustness beyond Isoperimetry

no code implementations23 Feb 2022 Yihan Wu, Heng Huang, Hongyang Zhang

We prove a Lipschitzness lower bound $\Omega(\sqrt{n/p})$ of the interpolating neural network with $p$ parameters on arbitrary data distributions.

Boosting Barely Robust Learners: A New Perspective on Adversarial Robustness

no code implementations11 Feb 2022 Avrim Blum, Omar Montasser, Greg Shakhnarovich, Hongyang Zhang

We present an oracle-efficient algorithm for boosting the adversarial robustness of barely robust learners.

Adversarial Robustness

Towards Transferable Unrestricted Adversarial Examples with Minimum Changes

1 code implementation4 Jan 2022 Fangcheng Liu, Chao Zhang, Hongyang Zhang

Extensive experiments verify the effectiveness of our framework on balancing imperceptibility and transferability of the crafted adversarial examples.

Adversarial Attack

Towards Transferable Adversarial Perturbations with Minimum Norm

no code implementations ICML Workshop AML 2021 Fangcheng Liu, Chao Zhang, Hongyang Zhang

In this work, we propose a \emph{geometry-aware framework} to generate transferable adversarial perturbation with minimum norm for each input.

Model Selection

Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning

2 code implementations21 Jan 2021 Lang Huang, Chao Zhang, Hongyang Zhang

We propose self-adaptive training -- a unified training algorithm that dynamically calibrates and enhances training processes by model predictions without incurring an extra computational cost -- to advance both supervised and self-supervised learning of deep neural networks.

Representation Learning Self-Supervised Learning

Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients

1 code implementation28 Sep 2020 Yifei Huang, Yaodong Yu, Hongyang Zhang, Yi Ma, Yuan YAO

Even replacing only the first layer of a ResNet by such a ODE block can exhibit further improvement in robustness, e. g., under PGD-20 ($\ell_\infty=0. 031$) attack on CIFAR-10 dataset, it achieves 91. 57\% and natural accuracy and 62. 35\% robust accuracy, while a counterpart architecture of ResNet trained with TRADES achieves natural and robust accuracy 76. 29\% and 45. 24\%, respectively.

Adversarial Defense Adversarial Robustness

A Closer Look at Accuracy vs. Robustness

1 code implementation NeurIPS 2020 Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov, Kamalika Chaudhuri

Current methods for training robust networks lead to a drop in test accuracy, which has led prior works to posit that a robustness-accuracy tradeoff may be inevitable in deep learning.

Self-Adaptive Training: beyond Empirical Risk Minimization

4 code implementations NeurIPS 2020 Lang Huang, Chao Zhang, Hongyang Zhang

We propose self-adaptive training---a new training algorithm that dynamically corrects problematic training labels by model predictions without incurring extra computational cost---to improve generalization of deep learning for potentially corrupted training data.

General Classification Test

Random Smoothing Might be Unable to Certify $\ell_\infty$ Robustness for High-Dimensional Images

1 code implementation10 Feb 2020 Avrim Blum, Travis Dick, Naren Manoj, Hongyang Zhang

We show a hardness result for random smoothing to achieve certified adversarial robustness against attacks in the $\ell_p$ ball of radius $\epsilon$ when $p>2$.

Adversarial Robustness

Optimal Analysis of Subset-Selection Based L_p Low-Rank Approximation

no code implementations NeurIPS 2019 Chen Dan, Hong Wang, Hongyang Zhang, Yuchen Zhou, Pradeep K. Ravikumar

We show that this algorithm has an approximation ratio of $O((k+1)^{1/p})$ for $1\le p\le 2$ and $O((k+1)^{1-1/p})$ for $p\ge 2$.

Design and Interpretation of Universal Adversarial Patches in Face Detection

no code implementations ECCV 2020 Xiao Yang, Fangyun Wei, Hongyang Zhang, Jun Zhu

We consider universal adversarial patches for faces -- small visual elements whose addition to a face image reliably destroys the performance of face detectors.

Face Detection

Optimal Analysis of Subset-Selection Based L_p Low Rank Approximation

no code implementations30 Oct 2019 Chen Dan, Hong Wang, Hongyang Zhang, Yuchen Zhou, Pradeep Ravikumar

We show that this algorithm has an approximation ratio of $O((k+1)^{1/p})$ for $1\le p\le 2$ and $O((k+1)^{1-1/p})$ for $p\ge 2$.

Efficient Symmetric Norm Regression via Linear Sketching

no code implementations NeurIPS 2019 Zhao Song, Ruosong Wang, Lin F. Yang, Hongyang Zhang, Peilin Zhong

When the loss function is a general symmetric norm, our algorithm produces a $\sqrt{d} \cdot \mathrm{polylog} n \cdot \mathrm{mmc}(\ell)$-approximate solution in input-sparsity time, where $\mathrm{mmc}(\ell)$ is a quantity related to the symmetric norm under consideration.

regression

Theoretically Principled Trade-off between Robustness and Accuracy

8 code implementations24 Jan 2019 Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, Michael. I. Jordan

We identify a trade-off between robustness and accuracy that serves as a guiding principle in the design of defenses against adversarial examples.

Adversarial Attack Adversarial Defense +2

Testing Matrix Rank, Optimally

no code implementations18 Oct 2018 Maria-Florina Balcan, Yi Li, David P. Woodruff, Hongyang Zhang

This improves upon the previous $O(d^2/\epsilon^2)$ bound (SODA'03), and bypasses an $\Omega(d^2/\epsilon^2)$ lower bound of (KDD'14) which holds if the algorithm is required to read a submatrix.

Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex

1 code implementation6 Jun 2018 Hongyang Zhang, Junru Shao, Ruslan Salakhutdinov

We show that one cause for such success is due to the fact that the multi-branch architecture is less non-convex in terms of duality gap.

Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations

no code implementations26 Dec 2017 Yuanzhi Li, Tengyu Ma, Hongyang Zhang

We show that the gradient descent algorithm provides an implicit regularization effect in the learning of over-parameterized matrix factorization models and one-hidden-layer neural networks with quadratic activations.

Noise-Tolerant Interactive Learning Using Pairwise Comparisons

no code implementations NeurIPS 2017 Yichong Xu, Hongyang Zhang, Kyle Miller, Aarti Singh, Artur Dubrawski

We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive.

Differentially Private Clustering in High-Dimensional Euclidean Spaces

no code implementations ICML 2017 Maria-Florina Balcan, Travis Dick, YIngyu Liang, Wenlong Mou, Hongyang Zhang

We study the problem of clustering sensitive data while preserving the privacy of individuals represented in the dataset, which has broad applications in practical machine learning and data analysis tasks.

Clustering Vocal Bursts Intensity Prediction

Regression Phalanxes

no code implementations3 Jul 2017 Hongyang Zhang, William J. Welch, Ruben H. Zamar

Tomal et al. (2015) introduced the notion of "phalanxes" in the context of rare-class detection in two-class classification problems.

Clustering Drug Discovery +2

Matrix Completion and Related Problems via Strong Duality

no code implementations27 Apr 2017 Maria-Florina Balcan, YIngyu Liang, David P. Woodruff, Hongyang Zhang

This work studies the strong duality of non-convex matrix factorization problems: we show that under certain dual conditions, these problems and its dual have the same optimum.

Matrix Completion

Noise-Tolerant Interactive Learning from Pairwise Comparisons

no code implementations19 Apr 2017 Yichong Xu, Hongyang Zhang, Aarti Singh, Kyle Miller, Artur Dubrawski

We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive.

Sample and Computationally Efficient Learning Algorithms under S-Concave Distributions

no code implementations NeurIPS 2017 Maria-Florina Balcan, Hongyang Zhang

In this work, we introduce new convex geometry tools to study the properties of $s$-concave distributions and use these properties to provide bounds on quantities of interest to learning including the probability of disagreement between two halfspaces, disagreement outside a band, and the disagreement coefficient.

Active Learning

Noise-Tolerant Life-Long Matrix Completion via Adaptive Sampling

no code implementations NeurIPS 2016 Maria-Florina Balcan, Hongyang Zhang

For this problem, we present an algorithm that returns a matrix of a small error, with sample complexity almost as small as the best prior results in the noiseless case.

Matrix Completion

Completing Low-Rank Matrices with Corrupted Samples from Few Coefficients in General Basis

no code implementations25 Jun 2015 Hongyang Zhang, Zhouchen Lin, Chao Zhang

As an application, we also find that the solutions to extended robust Low-Rank Representation and to our extended robust MC are mutually expressible, so both our theory and algorithm can be applied to the subspace clustering problem with missing values under certain conditions.

Clustering Matrix Completion

Relations among Some Low Rank Subspace Recovery Models

no code implementations6 Dec 2014 Hongyang Zhang, Zhouchen Lin, Chao Zhang, Junbin Gao

More specifically, we discover that once a solution to one of the models is obtained, we can obtain the solutions to other models in closed-form formulations.

Multi-rank Sparse Hierarchical Clustering

no code implementations2 Sep 2014 Hongyang Zhang, Ruben H. Zamar

There has been a surge in the number of large and flat data sets - data sets containing a large number of features and a relatively small number of observations - due to the growing ability to collect and store information in medical research and other fields.

Clustering feature selection

A Counterexample for the Validity of Using Nuclear Norm as a Convex Surrogate of Rank

no code implementations23 Apr 2013 Hongyang Zhang, Zhouchen Lin, Chao Zhang

For several rank minimization problems, such a replacement has been theoretically proven to be valid, i. e., the solution to nuclear norm minimization problem is also the solution to rank minimization problem.

valid

Cannot find the paper you are looking for? You can Submit a new open access paper.