Search Results for author: Ziteng Sun

Found 25 papers, 3 papers with code

Context Aware Local Differential Privacy

no code implementations ICML 2020 Jayadev Acharya, Kallista Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun

The original definition of LDP assumes that all the elements in the data domain are equally sensitive.

Asymptotics of Language Model Alignment

no code implementations2 Apr 2024 Joy Qiping Yang, Salman Salamatian, Ziteng Sun, Ananda Theertha Suresh, Ahmad Beirami

The goal of language model alignment is to alter $p$ to a new distribution $\phi$ that results in a higher expected reward while keeping $\phi$ close to $p.$ A popular alignment method is the KL-constrained reinforcement learning (RL), which chooses a distribution $\phi_\Delta$ that maximizes $E_{\phi_{\Delta}} r(y)$ subject to a relative entropy constraint $KL(\phi_\Delta || p) \leq \Delta.$ Another simple alignment method is best-of-$N$, where $N$ samples are drawn from $p$ and one with highest reward is selected.

Language Modelling Reinforcement Learning (RL)

Optimal Block-Level Draft Verification for Accelerating Speculative Decoding

no code implementations15 Mar 2024 Ziteng Sun, Jae Hun Ro, Ahmad Beirami, Ananda Theertha Suresh

To the best of our knowledge, our work is the first to establish improvement over speculative decoding through a better draft verification algorithm.

SpecTr: Fast Speculative Decoding via Optimal Transport

no code implementations NeurIPS 2023 Ziteng Sun, Ananda Theertha Suresh, Jae Hun Ro, Ahmad Beirami, Himanshu Jain, Felix Yu

We show that the optimal draft selection algorithm (transport plan) can be computed via linear programming, whose best-known runtime is exponential in $k$.

Language Modelling Large Language Model

The importance of feature preprocessing for differentially private linear optimization

no code implementations19 Jul 2023 Ziteng Sun, Ananda Theertha Suresh, Aditya Krishna Menon

Training machine learning models with differential privacy (DP) has received increasing interest in recent years.

Image Classification

Subset-Based Instance Optimality in Private Estimation

no code implementations1 Mar 2023 Travis Dick, Alex Kulesza, Ziteng Sun, Ananda Theertha Suresh

We propose a new definition of instance optimality for differentially private estimation algorithms.

Concentration Bounds for Discrete Distribution Estimation in KL Divergence

no code implementations14 Feb 2023 Clément L. Canonne, Ziteng Sun, Ananda Theertha Suresh

We study the problem of discrete distribution estimation in KL divergence and provide concentration bounds for the Laplace estimator.

Discrete Distribution Estimation under User-level Local Differential Privacy

no code implementations7 Nov 2022 Jayadev Acharya, YuHan Liu, Ziteng Sun

Perhaps surprisingly, we show that in suitable parameter regimes, having $m$ samples per user is equivalent to having $m$ times more users, each with only one sample.

The Role of Interactivity in Structured Estimation

no code implementations14 Mar 2022 Jayadev Acharya, Clément L. Canonne, Ziteng Sun, Himanshu Tyagi

Without sparsity assumptions, it has been established that interactivity cannot improve the minimax rates of estimation under these information constraints.

Compressive Sensing

Correlated quantization for distributed mean estimation and optimization

no code implementations9 Mar 2022 Ananda Theertha Suresh, Ziteng Sun, Jae Hun Ro, Felix Yu

We show that applying the proposed protocol as sub-routine in distributed optimization algorithms leads to better convergence rates.

Distributed Optimization Quantization

Distributed Estimation with Multiple Samples per User: Sharp Rates and Phase Transition

no code implementations NeurIPS 2021 Jayadev Acharya, Clement Canonne, YuHan Liu, Ziteng Sun, Himanshu Tyagi

We obtain tight minimax rates for the problem of distributed estimation of discrete distributions under communication constraints, where $n$ users observing $m $ samples each can broadcast only $\ell$ bits.

Robust Testing and Estimation under Manipulation Attacks

no code implementations21 Apr 2021 Jayadev Acharya, Ziteng Sun, Huanyu Zhang

We consider both the "centralized setting" and the "distributed setting with information constraints" including communication and local privacy (LDP) constraints.

Learning with User-Level Privacy

no code implementations NeurIPS 2021 Daniel Levy, Ziteng Sun, Kareem Amin, Satyen Kale, Alex Kulesza, Mehryar Mohri, Ananda Theertha Suresh

We show that for high-dimensional mean estimation, empirical risk minimization with smooth losses, stochastic convex optimization, and learning hypothesis classes with finite metric entropy, the privacy cost decreases as $O(1/\sqrt{m})$ as users provide more samples.

Estimating Sparse Discrete Distributions Under Local Privacy and Communication Constraints

no code implementations30 Oct 2020 Jayadev Acharya, Peter Kairouz, YuHan Liu, Ziteng Sun

We consider the problem of estimating sparse discrete distributions under local differential privacy (LDP) and communication constraints.

Interactive Inference under Information Constraints

no code implementations21 Jul 2020 Jayadev Acharya, Clément L. Canonne, Yu-Han Liu, Ziteng Sun, Himanshu Tyagi

We study the role of interactivity in distributed statistical inference under information constraints, e. g., communication constraints and local differential privacy.

Density Estimation

Differentially Private Assouad, Fano, and Le Cam

no code implementations14 Apr 2020 Jayadev Acharya, Ziteng Sun, Huanyu Zhang

The technical component of our paper relates coupling between distributions to the sample complexity of estimation under differential privacy.


Can You Really Backdoor Federated Learning?

no code implementations18 Nov 2019 Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, H. Brendan McMahan

This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task.

Federated Learning

Estimating Entropy of Distributions in Constant Space

no code implementations NeurIPS 2019 Jayadev Acharya, Sourbh Bhadane, Piotr Indyk, Ziteng Sun

We consider the task of estimating the entropy of $k$-ary distributions from samples in the streaming model, where space is limited.

Context-Aware Local Differential Privacy

no code implementations31 Oct 2019 Jayadev Acharya, Keith Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun

Local differential privacy (LDP) is a strong notion of privacy for individual users that often comes at the expense of a significant drop in utility.

Domain Compression and its Application to Randomness-Optimal Distributed Goodness-of-Fit

no code implementations20 Jul 2019 Jayadev Acharya, Clément L. Canonne, Yanjun Han, Ziteng Sun, Himanshu Tyagi

We study goodness-of-fit of discrete distributions in the distributed setting, where samples are divided between multiple users who can only release a limited amount of information about their samples due to various information constraints.

Communication Complexity in Locally Private Distribution Estimation and Heavy Hitters

no code implementations28 May 2019 Jayadev Acharya, Ziteng Sun

We consider the problems of distribution estimation and heavy hitter (frequency) estimation under privacy and communication constraints.

INSPECTRE: Privately Estimating the Unseen

1 code implementation ICML 2018 Jayadev Acharya, Gautam Kamath, Ziteng Sun, Huanyu Zhang

We develop differentially private methods for estimating various distributional properties.

Hadamard Response: Estimating Distributions Privately, Efficiently, and with Little Communication

3 code implementations13 Feb 2018 Jayadev Acharya, Ziteng Sun, Huanyu Zhang

All previously known sample optimal algorithms require linear (in $k$) communication from each user in the high privacy regime $(\varepsilon=O(1))$, and run in time that grows as $n\cdot k$, which can be prohibitive for large domain size $k$.

Differentially Private Testing of Identity and Closeness of Discrete Distributions

no code implementations NeurIPS 2018 Jayadev Acharya, Ziteng Sun, Huanyu Zhang

We propose a general framework to establish lower bounds on the sample complexity of statistical tasks under differential privacy.

Cannot find the paper you are looking for? You can Submit a new open access paper.