Search Results for author:
Found 2 papers, 1 papers with code
DC-SGD: Differentially Private SGD with Dynamic Clipping through Gradient Norm Distribution Estimation
In this paper, we propose Dynamic Clipping DP-SGD (DC-SGD), a framework that leverages differentially private histograms to estimate gradient norm distributions and dynamically adjust the clipping threshold C. Our framework includes two novel mechanisms: DC-SGD-P and DC-SGD-E. DC-SGD-P adjusts the clipping threshold based on a percentile of gradient norms, while DC-SGD-E minimizes the expected squared error of gradients to optimize C. These dynamic adjustments significantly reduce the burden of hyperparameter tuning C. The extensive experiments on various deep learning tasks, including image classification and natural language processing, show that our proposed dynamic algorithms achieve up to 9 times acceleration on hyperparameter tuning than DP-SGD.
R.R.: Unveiling LLM Training Privacy through Recollection and Ranking
In the second stage, we design a new criterion to score each PII candidate and rank them.
