Search Results for author:
Found 17 papers, 8 papers with code
DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theory
Quantizing the weights of a neural network has two steps: (1) Finding a good low bit-complexity representation for weights (which we call the quantization grid) and (2) Rounding the original weights to values in the quantization grid.
Efficiently Computing Similarities to Private Datasets
We abstract out this common subroutine and study the following fundamental algorithmic problem: Given a similarity function $f$ and a large high-dimensional private dataset $X \subset \mathbb{R}^d$, output a differentially private (DP) data structure which approximates $\sum_{x \in X} f(x, y)$ for any query $y$.
Differentially Private Synthetic Data via Foundation Model APIs 2: Text
Lin et al. (2024) recently introduced the Private Evolution (PE) algorithm to generate DP synthetic images with only API access to diffusion models.
Privately Aligning Language Models with Reinforcement Learning
Positioned between pre-training and user deployment, aligning large language models (LLMs) through reinforcement learning (RL) has emerged as a prevailing strategy for training instruction following-models such as ChatGPT.
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
To reduce the compute time overhead of private learning, we show that \emph{per-layer clipping}, where the gradient of each neural network layer is clipped separately, allows clipping to be performed in conjunction with backpropagation in differentially private optimization.
Unveiling Transformers with LEGO: a synthetic reasoning task
We study how the trained models eventually succeed at the task, and in particular, we manage to understand some of the attention heads as well as how the information flows in the network.
Differentially Private Model Compression
Recent papers have shown that large pre-trained language models (LLMs) such as BERT, GPT-2 can be fine-tuned on private data to achieve performance comparable to non-private models for many downstream Natural Language Processing (NLP) tasks while simultaneously guaranteeing differential privacy.
Differentially Private Fine-tuning of Language Models
For example, on the MNLI dataset we achieve an accuracy of $87. 8\%$ using RoBERTa-Large and $83. 5\%$ using RoBERTa-Base with a privacy budget of $\epsilon = 6. 7$.
Faster Kernel Matrix Algebra via Density Estimation
In particular, we consider estimating the sum of kernel matrix entries, along with its top eigenvalue and eigenvector.
Data-to-text Generation by Splicing Together Nearest Neighbors
We propose to tackle data-to-text generation tasks by directly splicing together retrieved segments of text from "neighbor" source-target pairs.
Impossibility Results for Grammar-Compressed Linear Algebra
In this paper we consider lossless compression schemes, and ask if we can run our computations on the compressed data as efficiently as if the original data was that small.
Active Local Learning
In particular, the number of label queries should be independent of the complexity of $H$, and the function $h$ should be well-defined, independent of $x$.
Space and Time Efficient Kernel Density Estimation in High Dimensions
We instantiate our framework with the Laplacian and Exponential kernels, two popular kernels which possess the aforementioned property.
Scalable Nearest Neighbor Search for Optimal Transport
Our extensive experiments, on real-world text and image datasets, show that Flowtree improves over various baselines and existing methods in either running time or accuracy.
Data Structures and Algorithms
Scalable Fair Clustering
In the fair variant of $k$-median, the points are colored, and the goal is to minimize the same average distance objective while ensuring that all clusters have an "approximately equal" number of points of each color.
Improving Viterbi is Hard: Better Runtimes Imply Faster Clique Algorithms
The classic algorithm of Viterbi computes the most likely path in a Hidden Markov Model (HMM) that results in a given sequence of observations.
On the Fine-Grained Complexity of Empirical Risk Minimization: Kernel Methods and Neural Networks
We also give similar hardness results for computing the gradient of the empirical loss, which is the main computational burden in many non-convex learning tasks.
