DC-SGD: Differentially Private SGD with Dynamic Clipping through Gradient Norm Distribution Estimation

no code implementations • 29 Mar 2025 • Chengkun Wei, Weixian Li, Chen Gong, Wenzhi Chen

In this paper, we propose Dynamic Clipping DP-SGD (DC-SGD), a framework that leverages differentially private histograms to estimate gradient norm distributions and dynamically adjust the clipping threshold C. Our framework includes two novel mechanisms: DC-SGD-P and DC-SGD-E. DC-SGD-P adjusts the clipping threshold based on a percentile of gradient norms, while DC-SGD-E minimizes the expected squared error of gradients to optimize C. These dynamic adjustments significantly reduce the burden of hyperparameter tuning C. The extensive experiments on various deep learning tasks, including image classification and natural language processing, show that our proposed dynamic algorithms achieve up to 9 times acceleration on hyperparameter tuning than DP-SGD.

Deep Learning Image Classification +2

Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation

1 code implementation • 11 Mar 2025 • Wenlong Meng, Fan Zhang, Wendao Yao, Zhenyuan Guo, Yuwei Li, Chengkun Wei, Wenzhi Chen

Our experiments show that DIA achieves state-of-the-art attack success rates on recent LLMs, including Llama-3. 1 and GPT-4o.

R.R.: Unveiling LLM Training Privacy through Recollection and Ranking

1 code implementation • 18 Feb 2025 • Wenlong Meng, Zhenyuan Guo, Lenan Wu, Chen Gong, Wenyan Liu, Weixian Li, Chengkun Wei, Wenzhi Chen

In the second stage, we design a new criterion to score each PII candidate and rank them.

Memorization

Be Cautious When Merging Unfamiliar LLMs: A Phishing Model Capable of Stealing Privacy

1 code implementation • 17 Feb 2025 • Zhenyuan Guo, Yi Shi, Wenlong Meng, Chen Gong, Chengkun Wei, Wenzhi Chen

Specifically, we propose PhiMM, a privacy attack approach that trains a phishing model capable of stealing privacy using a crafted privacy phishing instruction dataset.

LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors

1 code implementation • 26 Aug 2023 • Chengkun Wei, Wenlong Meng, Zhikun Zhang, Min Chen, Minghu Zhao, Wenjing Fang, Lei Wang, Zihui Zhang, Wenzhi Chen

Instead of directly inverting the triggers, LMSanitator aims to invert the predefined attack vectors (pretrained models' output when the input is embedded with triggers) of the task-agnostic backdoors, which achieves much better convergence performance and backdoor detection accuracy.

DPMLBench: Holistic Evaluation of Differentially Private Machine Learning

1 code implementation • 10 May 2023 • Chengkun Wei, Minghu Zhao, Zhikun Zhang, Min Chen, Wenlong Meng, Bo Liu, Yuan Fan, Wenzhi Chen

We also explore some improvements that can maintain model utility and defend against MIAs more effectively.

Image Classification