Search Results for author:
Found 18 papers, 6 papers with code
PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels
For context lengths of 32k and GPT-2 style models, our model achieves a 2. 5-4x speedup in training compared to FlashAttention, with no observed degradation in quality across our experiments.
Differentially Private Clustering in Data Streams
In this work, we provide the first differentially private streaming algorithms for $k$-means and $k$-median clustering of $d$-dimensional Euclidean data points over a stream with length at most $T$ using $poly(k, d,\log(T))$ space to achieve a constant multiplicative error and a $poly(k, d,\log(T))$ additive error.
Measuring Re-identification Risk
In this work, we present a new theoretical framework to measure re-identification risk in such user representations.
Stars: Tera-Scale Graph Building for Clustering and Graph Learning
We evaluate the performance of Stars for clustering and graph learning, and demonstrate 10~1000-fold improvements in pairwise similarity comparisons compared to different baselines, and 2~10-fold improvement in running time without quality loss.
Differentially Private Graph Learning via Sensitivity-Bounded Personalized PageRank
Personalized PageRank (PPR) is a fundamental tool in unsupervised learning of graph representations such as node ranking, labeling, and graph embedding.
Planning with General Objective Functions: Going Beyond Total Rewards
Standard sequential decision-making paradigms aim to maximize the cumulative reward when interacting with the unknown environment., i. e., maximize $\sum_{h = 1}^H r_h$ where $H$ is the planning horizon.
Average Case Column Subset Selection for Entrywise $\ell_1$-Norm Loss
entries drawn from any distribution $\mu$ for which the $(1+\gamma)$-th moment exists, for an arbitrarily small constant $\gamma > 0$, then it is possible to obtain a $(1+\epsilon)$-approximate column subset selection to the entrywise $\ell_1$-norm in nearly linear time.
Average Case Column Subset Selection for Entrywise \ell_1-Norm Loss
entries drawn from any distribution $\mu$ for which the $(1+\gamma)$-th moment exists, for an arbitrarily small constant $\gamma > 0$, then it is possible to obtain a $(1+\epsilon)$-approximate column subset selection to the entrywise $\ell_1$-norm in nearly linear time.
Efficient Symmetric Norm Regression via Linear Sketching
When the loss function is a general symmetric norm, our algorithm produces a $\sqrt{d} \cdot \mathrm{polylog} n \cdot \mathrm{mmc}(\ell)$-approximate solution in input-sparsity time, where $\mathrm{mmc}(\ell)$ is a quantity related to the symmetric norm under consideration.
Enhancing Adversarial Defense by k-Winners-Take-All
In all cases, the robustness of k-WTA networks outperforms that of traditional networks under white-box attacks.
Rethinking Generative Mode Coverage: A Pointwise Guaranteed Approach
The conventional wisdom to this end is by reducing through training a statistical distance (such as $f$-divergence) between the generated distribution and provided data distribution.
Towards a Zero-One Law for Column Subset Selection
Our approximation algorithms handle functions which are not even scale-invariant, such as the Huber loss function, which we show have very different structural properties than $\ell_p$-norms, e. g., one can show the lack of scale-invariance causes any column subset selection algorithm to provably require a $\sqrt{\log n}$ factor larger number of columns than $\ell_p$-norms; nevertheless we design the first efficient column subset selection algorithms for such error measures.
Subspace Embedding and Linear Regression with Orlicz Norm
An Orlicz norm is parameterized by a non-negative convex function $G:\mathbb{R}_+\rightarrow\mathbb{R}_+$ with $G(0)=0$: the Orlicz norm of a vector $x\in\mathbb{R}^n$ is defined as $ \|x\|_G=\inf\left\{\alpha>0\large\mid\sum_{i=1}^n G(|x_i|/\alpha)\leq 1\right\}.
BourGAN: Generative Networks with Metric Embeddings
This paper addresses the mode collapse for generative adversarial networks (GANs).
Nearly Optimal Dynamic $k$-Means Clustering for High-Dimensional Data
We consider the $k$-means clustering problem in the dynamic streaming setting, where points from a discrete Euclidean space $\{1, 2, \ldots, \Delta\}^d$ can be dynamically inserted to or deleted from the dataset.
Relative Error Tensor Low Rank Approximation
Despite the success on obtaining relative error low rank approximations for matrices, no such results were known for tensors.
Low Rank Approximation with Entrywise $\ell_1$-Norm Error
We give the first provable approximation algorithms for $\ell_1$-low rank approximation, showing that it is possible to achieve approximation factor $\alpha = (\log d) \cdot \mathrm{poly}(k)$ in $\mathrm{nnz}(A) + (n+d) \mathrm{poly}(k)$ time, where $\mathrm{nnz}(A)$ denotes the number of non-zero entries of $A$.
Distributed Low Rank Approximation of Implicit Functions of a Matrix
For example, each of $s$ servers may have an $n \times d$ matrix $A^t$, and we may be interested in computing a low rank approximation to $A = f(\sum_{t=1}^s A^t)$, where $f$ is a function which is applied entrywise to the matrix $\sum_{t=1}^s A^t$.
