Search Results for author:
Found 71 papers, 20 papers with code
Predicting quantum channels over general product distributions
Previously, Huang, Chen, and Preskill proved a surprising result that even if $E$ is arbitrary, this task can be solved in time roughly $n^{O(\log(1/\epsilon))}$, where $\epsilon$ is the target prediction error.
Optimal high-precision shadow estimation
We give the first tight sample complexity bounds for shadow tomography and classical shadows in the regime where the target error is below some sufficiently small inverse polynomial in the dimension of the Hilbert space.
Black-Box $k$-to-$1$-PCA Reductions: Theory and Applications
For an alternative well-studied approximation notion we term cPCA (correlation PCA), we tightly characterize the parameter regimes where deflation methods are feasible.
An optimal tradeoff between entanglement and copy complexity for state tomography
In this work, we study tomography in the natural setting where one can make measurements of $t$ copies at a time.
KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval
Motivated by rising concerns around factual incorrectness and hallucinations of LLMs, we present KITAB, a new dataset for measuring constraint satisfaction abilities of language models.
Matrix Completion in Almost-Verification Time
In the well-studied setting where $\mathbf{M}$ has incoherent row and column spans, our algorithms complete $\mathbf{M}$ to high precision from $mr^{2+o(1)}$ observations in $mr^{3 + o(1)}$ time (omitting logarithmic factors in problem parameters), improving upon the prior state-of-the-art [JN15] which used $\approx mr^5$ samples and $\approx mr^7$ time.
Automatic Prompt Optimization with "Gradient Descent" and Beam Search
Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort.
Query lower bounds for log-concave sampling
Log-concave sampling has witnessed remarkable algorithmic advances in recent years, but the corresponding problem of proving lower bounds for this task has remained elusive, with lower bounds previously known only in dimension one.
REAP: A Large-Scale Realistic Adversarial Patch Benchmark
In this work, we propose the REAP (REalistic Adversarial Patch) benchmark, a digital benchmark that allows the user to evaluate patch attacks on real images, and under real-world conditions.
Controlling Commercial Cooling Systems Using Reinforcement Learning
This paper is a technical overview of DeepMind and Google's recent work on reinforcement learning for controlling commercial cooling systems.
The Complexity of NISQ
The recent proliferation of NISQ devices has made it imperative to understand their computational power.
Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
We provide theoretical convergence guarantees for score-based generative models (SGMs) such as denoising diffusion probabilistic models (DDPMs), which constitute the backbone of large-scale real-world generative models such as DALL$\cdot$E 2.
When Does Adaptivity Help for Quantum State Learning?
We give an adaptive algorithm that outputs a state which is $\gamma$-close in infidelity to $\rho$ using only $\tilde{O}(d^3/\gamma)$ copies, which is optimal for incoherent measurements.
Learning (Very) Simple Generative Models Is Hard
Motivated by the recent empirical successes of deep generative models, we study the computational complexity of the following unsupervised learning problem.
Tight Bounds for Quantum State Certification with Incoherent Measurements
When $\sigma$ is the maximally mixed state $\frac{1}{d} I_d$, this is known as mixedness testing.
Learning Polynomial Transformations
Our first main result is a polynomial-time algorithm for learning quadratic transformations of Gaussians in a smoothed setting.
Semi-Random Sparse Recovery in Nearly-Linear Time
We design a new iterative method tailored to the geometry of sparse recovery which is provably robust to our semi-random model.
Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs
Arguably the most fundamental question in the theory of generative adversarial networks (GANs) is to understand to what extent GANs can actually learn the underlying distribution.
On Distinctive Properties of Universal Perturbations
We identify properties of universal adversarial perturbations (UAPs) that distinguish them from standard adversarial perturbations.
Clustering Mixtures with Almost Optimal Separation in Polynomial Time
We give the first algorithm which runs in polynomial time, and which almost matches this guarantee.
Quantum advantage in learning from experiments
Quantum technology has the potential to revolutionize how we acquire and process experimental data to learn about the physical world.
Exponential separations between learning with and without quantum memory
We study the power of quantum memory for learning properties of quantum systems and dynamics, which is of great importance in physics and chemistry.
A Hierarchy for Replica Quantum Advantage
We prove that given the ability to make entangled measurements on at most $k$ replicas of an $n$-qubit state $\rho$ simultaneously, there is a property of $\rho$ which requires at least order $2^n$ measurements to learn.
The Price of Tolerance in Distribution Testing
Specifically, we show the sample complexity to be \[\tilde \Theta\left(\frac{\sqrt{n}}{\varepsilon_2^{2}} + \frac{n}{\log n} \cdot \max \left\{\frac{\varepsilon_1}{\varepsilon_2^2},\left(\frac{\varepsilon_1}{\varepsilon_2^2}\right)^{\!\! 2}\right\}\right),\] providing a smooth tradeoff between the two previously known cases.
Robust Regression Revisited: Acceleration and Improved Estimation Rates
For the general case of smooth GLMs (e. g. logistic regression), we show that the robust gradient descent framework of Prasad et.
Clustering Mixture Models in Almost-Linear Time via List-Decodable Mean Estimation
We leverage this result, together with additional techniques, to obtain the first almost-linear time algorithms for clustering mixtures of $k$ separated well-behaved distributions, nearly-matching the statistical guarantees of spectral methods.
Robust Model Selection and Nearly-Proper Learning for GMMs
Given $\textsf{poly}(k/\epsilon)$ samples from a distribution that is $\epsilon$-close in TV distance to a GMM with $k$ components, we can construct a GMM with $\widetilde{O}(k)$ components that approximates the distribution to within $\widetilde{O}(\epsilon)$ in $\textsf{poly}(k/\epsilon)$ time.
Toward Instance-Optimal State Certification With Incoherent Measurements
We revisit the basic problem of quantum state certification: given copies of unknown mixed state $\rho\in\mathbb{C}^{d\times d}$ and the description of a mixed state $\sigma$, decide whether $\sigma = \rho$ or $\|\sigma - \rho\|_{\mathsf{tr}} \ge \epsilon$.
Towards Machine Ethics with Language Models
We show how to assess a language model’s knowledge of basic concepts of morality.
Non-robust Features through the Lens of Universal Perturbations
We study universal adversarial perturbations and demonstrate that the above picture is more nuanced.
Byzantine-Resilient Non-Convex Stochastic Gradient Descent
We study adversary-resilient stochastic distributed optimization, in which $m$ machines can independently compute stochastic gradients, and cooperate to jointly optimize over their local objective functions.
RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning
We hope that our suite of benchmarks will increase the reproducibility of experiments and make it possible to study challenging tasks with a limited computational budget, thus making RL research both more systematic and more accessible across the community.
List-Decodable Mean Estimation in Nearly-PCA Time
Our algorithm runs in time $\widetilde{O}(ndk)$ for all $k = O(\sqrt{d}) \cup \Omega(d)$, where $n$ is the size of the dataset.
Statistical Query Algorithms and Low-Degree Tests Are Almost Equivalent
Researchers currently use a number of approaches to predict and substantiate information-computation gaps in high-dimensional statistical estimation problems.
Aligning AI With Shared Human Values
We show how to assess a language model's knowledge of basic concepts of morality.
Ranked #1 on
Average
on hendrycks2020ethics
Fast and Near-Optimal Diagonal Preconditioning
In this paper, we revisit the decades-old problem of how to best improve $\mathbf{A}$'s condition number by left or right diagonal rescaling.
Robust and Heavy-Tailed Mean Estimation Made Simple, via Regret Minimization
In this paper, we provide a meta-problem and a duality theorem that lead to a new unified view on robust and heavy-tailed mean estimation in high dimensions.
Security and Machine Learning in the Real World
Machine learning (ML) models deployed in many safety- and business-critical systems are vulnerable to exploitation through adversarial examples.
RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning
We hope that our suite of benchmarks will increase the reproducibility of experiments and make it possible to study challenging tasks with a limited computational budget, thus making RL research both more systematic and more accessible across the community.
Robust Gaussian Covariance Estimation in Nearly-Matrix Multiplication Time
Previous work of Cheng et al demonstrated an algorithm that, given $N = \Omega (d^2 / \varepsilon^2)$ samples, achieved a near-optimal error of $O(\varepsilon \log 1 / \varepsilon)$, and moreover, their algorithm ran in time $\widetilde{O}(T(N, d) \log \kappa / \mathrm{poly} (\varepsilon))$, where $T(N, d)$ is the time it takes to multiply a $d \times N$ matrix by its transpose, and $\kappa$ is the condition number of $\Sigma$.
Robust Sub-Gaussian Principal Component Analysis and Width-Independent Schatten Packing
We develop two methods for the following fundamental statistical task: given an $\epsilon$-corrupted set of $n$ samples from a $d$-dimensional sub-Gaussian distribution, return an approximate top eigenvector of the covariance matrix.
An empirical investigation of the challenges of real-world reinforcement learning
We believe that an approach that addresses our set of proposed challenges would be readily deployable in a large number of real world problems.
Efficient Algorithms for Multidimensional Segmented Regression
We study the fundamental problem of fixed design {\em multidimensional segmented regression}: Given noisy samples from a function $f$, promised to be piecewise linear on an unknown set of $k$ rectangles, we want to recover $f$ up to a desired accuracy in mean-squared error.
Learning Structured Distributions From Untrusted Batches: Faster and Simpler
We revisit the problem of learning from untrusted batches introduced by Qiao and Valiant [QV17].
Randomized Smoothing of All Shapes and Sizes
Randomized smoothing is the current state-of-the-art defense with provable robustness against $\ell_2$ adversarial attacks.
Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments
In this paper, we give the first algorithm for learning an MLR that runs in time which is sub-exponential in $k$.
Efficiently Learning Structured Distributions from Untrusted Batches
When $k = 1$ this is the standard robust univariate density estimation setting and it is well-understood that $\Omega (\epsilon)$ error is unavoidable.
Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier Detection
In robust mean estimation the goal is to estimate the mean $\mu$ of a distribution on $\mathbb{R}^d$ given $n$ independent samples, an $\varepsilon$-fraction of which have been corrupted by a malicious adversary.
Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
In this paper, we employ adversarial training to improve the performance of randomized smoothing.
Sample Efficient Toeplitz Covariance Estimation
In addition to results that hold for any Toeplitz $T$, we further study the important setting when $T$ is close to low-rank, which is often the case in practice.
Spectral Signatures in Backdoor Attacks
In this paper, we identify a new property of all known backdoor attacks, which we call \emph{spectral signatures}.
Twin-GAN -- Unpaired Cross-Domain Image Translation with Weight-Sharing GANs
We present a framework for translating unlabeled images from one domain into analog images in another domain.
Privately Learning High-Dimensional Distributions
We present novel, computationally efficient, and differentially private algorithms for two fundamental high-dimensional learning problems: learning a multivariate Gaussian and learning a product distribution over the Boolean hypercube in total variation distance.
Byzantine Stochastic Gradient Descent
This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of the $m$ machines which allegedly compute stochastic gradients every iteration, an $\alpha$-fraction are Byzantine, and can behave arbitrarily and adversarially.
Sever: A Robust Meta-Algorithm for Stochastic Optimization
In high dimensions, most machine learning methods are brittle to even a small fraction of structured outliers.
Fast and Sample Near-Optimal Algorithms for Learning Multidimensional Histograms
We give an algorithm for this learning problem that uses $n = \tilde{O}_d(k/\epsilon^2)$ samples and runs in time $\tilde{O}_d(n)$.
On the limitations of first order approximation in GAN dynamics
This suggests that such usage of the first order approximation of the discriminator, which is a de-facto standard in all the existing GAN dynamics, might be one of the factors that makes GAN training so challenging in practice.
Communication-Efficient Distributed Learning of Discrete Distributions
For the case of structured distributions, such as k-histograms and monotone distributions, we design distributed learning algorithms that achieve significantly better communication guarantees than the naive ones, and obtain tight upper and lower bounds in several regimes.
ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning
We examine training at reduced precision, both from a theoretical and practical perspective, and ask: is it possible to train models at end-to-end low precision with provable guarantees?
On the Limitations of First-Order Approximation in GAN Dynamics
While Generative Adversarial Networks (GANs) have demonstrated promising performance on multiple vision tasks, their learning dynamics are not yet well understood, both in theory and in practice.
The Power of Choice in Priority Scheduling
We answer this question, showing that this strategy provides surprisingly strong guarantees: Although the single-choice process, where we always insert and remove from a single randomly chosen queue, has degrading cost, going to infinity as we increase the number of steps, in the two choice process, the expected rank of a removed element is $O( n )$ while the expected worst-case cost is $O( n \log n )$.
Data Structures and Algorithms Distributed, Parallel, and Cluster Computing
Robustly Learning a Gaussian: Getting Optimal Error, Efficiently
We give robust estimators that achieve estimation error $O(\varepsilon)$ in the total variation distance, which is optimal up to a universal constant that is independent of the dimension.
Being Robust (in High Dimensions) Can Be Practical
Robust estimation is much more challenging in high dimensions than it is in one dimension: Most techniques either lead to intractable optimization problems or estimators that can tolerate only a tiny fraction of errors.
Robust Sparse Estimation Tasks in High Dimensions
In this paper we initiate the study of whether or not sparse estimation tasks can be performed efficiently in high dimensions, in the robust setting where an $\eps$-fraction of samples are corrupted adversarially.
The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning
When applied to linear models together with double sampling, we save up to another 1. 7x in data movement compared with uniform quantization.
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
In this paper, we propose Quantized SGD (QSGD), a family of compression schemes which allow the compression of gradient updates at each node, while guaranteeing convergence under standard assumptions.
Fast Algorithms for Segmented Regression
We study the fixed design segmented regression problem: Given noisy samples from a piecewise linear function $f$, we want to recover $f$ up to a desired accuracy in mean-squared error.
Robust Estimators in High Dimensions without the Computational Intractability
We study high-dimensional distribution learning in an agnostic setting where an adversary is allowed to arbitrarily corrupt an $\varepsilon$-fraction of the samples.
A Nearly Optimal and Agnostic Algorithm for Properly Learning a Mixture of k Gaussians, for any Constant k
One notion of learning a GMM is proper learning: here, the goal is to find a mixture of $k$ Gaussians $\mathcal{M}$ that is close to the density $f$ of the unknown distribution from which we draw samples.
Sample-Optimal Density Estimation in Nearly-Linear Time
Let $f$ be the density function of an arbitrary univariate distribution, and suppose that $f$ is $\mathrm{OPT}$-close in $L_1$-distance to an unknown piecewise polynomial function with $t$ interval pieces and degree $d$.
Lower Bounds for Exact Model Counting and Applications in Probabilistic Databases
The best current methods for exactly computing the number of satisfying assignments, or the satisfying probability, of Boolean formulas can be seen, either directly or indirectly, as building 'decision-DNNF' (decision decomposable negation normal form) representations of the input Boolean formulas.
