Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation

2 code implementations • 10 Oct 2023 • Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, Danqi Chen

Finally, we propose an effective alignment method that explores diverse generation strategies, which can reasonably reduce the misalignment rate under our attack.

Privacy Implications of Retrieval-Based Language Models

1 code implementation • 24 May 2023 • Yangsibo Huang, Samyak Gupta, Zexuan Zhong, Kai Li, Danqi Chen

Crucially, we find that $k$NN-LMs are more susceptible to leaking private information from their private datastore than parametric models.

Retrieval

Recovering Private Text in Federated Learning of Language Models

1 code implementation • 17 May 2022 • Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, Danqi Chen

For the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences.

Federated Learning Word Embeddings

Evaluating Gradient Inversion Attacks and Defenses in Federated Learning

1 code implementation • NeurIPS 2021 • Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, Sanjeev Arora

Gradient inversion attack (or input recovery from gradient) is an emerging threat to the security and privacy preservation of Federated learning, whereby malicious eavesdroppers or participants in the protocol can recover (partially) the clients' private data.

Federated Learning