Search Results for author: Jerry Li

Found 69 papers, 20 papers with code

Black-Box $k$-to-$1$-PCA Reductions: Theory and Applications

no code implementations6 Mar 2024 Arun Jambulapati, Syamantak Kumar, Jerry Li, Shourya Pandey, Ankit Pensia, Kevin Tian

The $k$-principal component analysis ($k$-PCA) problem is a fundamental algorithmic primitive that is widely-used in data analysis and dimensionality reduction applications.

Dimensionality Reduction

An optimal tradeoff between entanglement and copy complexity for state tomography

no code implementations26 Feb 2024 Sitan Chen, Jerry Li, Allen Liu

In this work, we study tomography in the natural setting where one can make measurements of $t$ copies at a time.

KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval

1 code implementation24 Oct 2023 Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Ghosh Peshawaria, Ranjita Naik, Besmira Nushi

Motivated by rising concerns around factual incorrectness and hallucinations of LLMs, we present KITAB, a new dataset for measuring constraint satisfaction abilities of language models.

Information Retrieval Retrieval

Matrix Completion in Almost-Verification Time

no code implementations7 Aug 2023 Jonathan A. Kelner, Jerry Li, Allen Liu, Aaron Sidford, Kevin Tian

In the well-studied setting where $\mathbf{M}$ has incoherent row and column spans, our algorithms complete $\mathbf{M}$ to high precision from $mr^{2+o(1)}$ observations in $mr^{3 + o(1)}$ time (omitting logarithmic factors in problem parameters), improving upon the prior state-of-the-art [JN15] which used $\approx mr^5$ samples and $\approx mr^7$ time.

Low-Rank Matrix Completion

Automatic Prompt Optimization with "Gradient Descent" and Beam Search

3 code implementations4 May 2023 Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, Michael Zeng

Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort.

Query lower bounds for log-concave sampling

no code implementations5 Apr 2023 Sinho Chewi, Jaume de Dios Pont, Jerry Li, Chen Lu, Shyam Narayanan

Log-concave sampling has witnessed remarkable algorithmic advances in recent years, but the corresponding problem of proving lower bounds for this task has remained elusive, with lower bounds previously known only in dimension one.

REAP: A Large-Scale Realistic Adversarial Patch Benchmark

1 code implementation ICCV 2023 Nabeel Hingun, Chawin Sitawarin, Jerry Li, David Wagner

In this work, we propose the REAP (REalistic Adversarial Patch) benchmark, a digital benchmark that allows the user to evaluate patch attacks on real images, and under real-world conditions.

The Complexity of NISQ

no code implementations13 Oct 2022 Sitan Chen, Jordan Cotler, Hsin-Yuan Huang, Jerry Li

The recent proliferation of NISQ devices has made it imperative to understand their computational power.

Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

no code implementations22 Sep 2022 Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, Anru R. Zhang

We provide theoretical convergence guarantees for score-based generative models (SGMs) such as denoising diffusion probabilistic models (DDPMs), which constitute the backbone of large-scale real-world generative models such as DALL$\cdot$E 2.

Denoising

When Does Adaptivity Help for Quantum State Learning?

no code implementations10 Jun 2022 Sitan Chen, Brice Huang, Jerry Li, Allen Liu, Mark Sellke

We give an adaptive algorithm that outputs a state which is $\gamma$-close in infidelity to $\rho$ using only $\tilde{O}(d^3/\gamma)$ copies, which is optimal for incoherent measurements.

Open-Ended Question Answering

Learning (Very) Simple Generative Models Is Hard

no code implementations31 May 2022 Sitan Chen, Jerry Li, Yuanzhi Li

Motivated by the recent empirical successes of deep generative models, we study the computational complexity of the following unsupervised learning problem.

Tight Bounds for Quantum State Certification with Incoherent Measurements

no code implementations14 Apr 2022 Sitan Chen, Brice Huang, Jerry Li, Allen Liu

When $\sigma$ is the maximally mixed state $\frac{1}{d} I_d$, this is known as mixedness testing.

Learning Polynomial Transformations

no code implementations8 Apr 2022 Sitan Chen, Jerry Li, Yuanzhi Li, Anru R. Zhang

Our first main result is a polynomial-time algorithm for learning quadratic transformations of Gaussians in a smoothed setting.

Tensor Decomposition

Semi-Random Sparse Recovery in Nearly-Linear Time

no code implementations8 Mar 2022 Jonathan A. Kelner, Jerry Li, Allen Liu, Aaron Sidford, Kevin Tian

We design a new iterative method tailored to the geometry of sparse recovery which is provably robust to our semi-random model.

Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs

no code implementations ICLR 2022 Sitan Chen, Jerry Li, Yuanzhi Li, Raghu Meka

Arguably the most fundamental question in the theory of generative adversarial networks (GANs) is to understand to what extent GANs can actually learn the underlying distribution.

On Distinctive Properties of Universal Perturbations

no code implementations31 Dec 2021 Sung Min Park, Kuo-An Wei, Kai Xiao, Jerry Li, Aleksander Madry

We identify properties of universal adversarial perturbations (UAPs) that distinguish them from standard adversarial perturbations.

Clustering Mixtures with Almost Optimal Separation in Polynomial Time

no code implementations1 Dec 2021 Jerry Li, Allen Liu

We give the first algorithm which runs in polynomial time, and which almost matches this guarantee.

Clustering

Quantum advantage in learning from experiments

1 code implementation1 Dec 2021 Hsin-Yuan Huang, Michael Broughton, Jordan Cotler, Sitan Chen, Jerry Li, Masoud Mohseni, Hartmut Neven, Ryan Babbush, Richard Kueng, John Preskill, Jarrod R. McClean

Quantum technology has the potential to revolutionize how we acquire and process experimental data to learn about the physical world.

A Hierarchy for Replica Quantum Advantage

no code implementations10 Nov 2021 Sitan Chen, Jordan Cotler, Hsin-Yuan Huang, Jerry Li

We prove that given the ability to make entangled measurements on at most $k$ replicas of an $n$-qubit state $\rho$ simultaneously, there is a property of $\rho$ which requires at least order $2^n$ measurements to learn.

Exponential separations between learning with and without quantum memory

no code implementations10 Nov 2021 Sitan Chen, Jordan Cotler, Hsin-Yuan Huang, Jerry Li

We study the power of quantum memory for learning properties of quantum systems and dynamics, which is of great importance in physics and chemistry.

Open-Ended Question Answering

The Price of Tolerance in Distribution Testing

no code implementations25 Jun 2021 Clément L. Canonne, Ayush Jain, Gautam Kamath, Jerry Li

Specifically, we show the sample complexity to be \[\tilde \Theta\left(\frac{\sqrt{n}}{\varepsilon_2^{2}} + \frac{n}{\log n} \cdot \max \left\{\frac{\varepsilon_1}{\varepsilon_2^2},\left(\frac{\varepsilon_1}{\varepsilon_2^2}\right)^{\!\! 2}\right\}\right),\] providing a smooth tradeoff between the two previously known cases.

Robust Regression Revisited: Acceleration and Improved Estimation Rates

no code implementations NeurIPS 2021 Arun Jambulapati, Jerry Li, Tselil Schramm, Kevin Tian

For the general case of smooth GLMs (e. g. logistic regression), we show that the robust gradient descent framework of Prasad et.

regression

Clustering Mixture Models in Almost-Linear Time via List-Decodable Mean Estimation

no code implementations16 Jun 2021 Ilias Diakonikolas, Daniel M. Kane, Daniel Kongsgaard, Jerry Li, Kevin Tian

We leverage this result, together with additional techniques, to obtain the first almost-linear time algorithms for clustering mixtures of $k$ separated well-behaved distributions, nearly-matching the statistical guarantees of spectral methods.

Clustering

Robust Model Selection and Nearly-Proper Learning for GMMs

no code implementations5 Jun 2021 Jerry Li, Allen Liu, Ankur Moitra

Given $\textsf{poly}(k/\epsilon)$ samples from a distribution that is $\epsilon$-close in TV distance to a GMM with $k$ components, we can construct a GMM with $\widetilde{O}(k)$ components that approximates the distribution to within $\widetilde{O}(\epsilon)$ in $\textsf{poly}(k/\epsilon)$ time.

Learning Theory Model Selection

Toward Instance-Optimal State Certification With Incoherent Measurements

no code implementations25 Feb 2021 Sitan Chen, Jerry Li, Ryan O'Donnell

We revisit the basic problem of quantum state certification: given copies of unknown mixed state $\rho\in\mathbb{C}^{d\times d}$ and the description of a mixed state $\sigma$, decide whether $\sigma = \rho$ or $\|\sigma - \rho\|_{\mathsf{tr}} \ge \epsilon$.

Non-robust Features through the Lens of Universal Perturbations

no code implementations1 Jan 2021 Sung Min Park, Kuo-An Wei, Kai Yuanqing Xiao, Jerry Li, Aleksander Madry

We study universal adversarial perturbations and demonstrate that the above picture is more nuanced.

Byzantine-Resilient Non-Convex Stochastic Gradient Descent

no code implementations ICLR 2021 Zeyuan Allen-Zhu, Faeze Ebrahimian, Jerry Li, Dan Alistarh

We study adversary-resilient stochastic distributed optimization, in which $m$ machines can independently compute stochastic gradients, and cooperate to jointly optimize over their local objective functions.

Distributed Optimization

RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning

1 code implementation NeurIPS 2020 Caglar Gulcehre, Ziyu Wang, Alexander Novikov, Thomas Paine, Sergio Gómez, Konrad Zolna, Rishabh Agarwal, Josh S. Merel, Daniel J. Mankowitz, Cosmin Paduraru, Gabriel Dulac-Arnold, Jerry Li, Mohammad Norouzi, Matthew Hoffman, Nicolas Heess, Nando de Freitas

We hope that our suite of benchmarks will increase the reproducibility of experiments and make it possible to study challenging tasks with a limited computational budget, thus making RL research both more systematic and more accessible across the community.

Offline RL reinforcement-learning +1

List-Decodable Mean Estimation in Nearly-PCA Time

no code implementations NeurIPS 2021 Ilias Diakonikolas, Daniel M. Kane, Daniel Kongsgaard, Jerry Li, Kevin Tian

Our algorithm runs in time $\widetilde{O}(ndk)$ for all $k = O(\sqrt{d}) \cup \Omega(d)$, where $n$ is the size of the dataset.

Clustering

Statistical Query Algorithms and Low-Degree Tests Are Almost Equivalent

no code implementations13 Sep 2020 Matthew Brennan, Guy Bresler, Samuel B. Hopkins, Jerry Li, Tselil Schramm

Researchers currently use a number of approaches to predict and substantiate information-computation gaps in high-dimensional statistical estimation problems.

Two-sample testing

Fast and Near-Optimal Diagonal Preconditioning

no code implementations4 Aug 2020 Arun Jambulapati, Jerry Li, Christopher Musco, Aaron Sidford, Kevin Tian

In this paper, we revisit the decades-old problem of how to best improve $\mathbf{A}$'s condition number by left or right diagonal rescaling.

Robust and Heavy-Tailed Mean Estimation Made Simple, via Regret Minimization

no code implementations NeurIPS 2020 Samuel B. Hopkins, Jerry Li, Fred Zhang

In this paper, we provide a meta-problem and a duality theorem that lead to a new unified view on robust and heavy-tailed mean estimation in high dimensions.

Security and Machine Learning in the Real World

no code implementations13 Jul 2020 Ivan Evtimov, Weidong Cui, Ece Kamar, Emre Kiciman, Tadayoshi Kohno, Jerry Li

Machine learning (ML) models deployed in many safety- and business-critical systems are vulnerable to exploitation through adversarial examples.

BIG-bench Machine Learning

RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning

2 code implementations24 Jun 2020 Caglar Gulcehre, Ziyu Wang, Alexander Novikov, Tom Le Paine, Sergio Gomez Colmenarejo, Konrad Zolna, Rishabh Agarwal, Josh Merel, Daniel Mankowitz, Cosmin Paduraru, Gabriel Dulac-Arnold, Jerry Li, Mohammad Norouzi, Matt Hoffman, Ofir Nachum, George Tucker, Nicolas Heess, Nando de Freitas

We hope that our suite of benchmarks will increase the reproducibility of experiments and make it possible to study challenging tasks with a limited computational budget, thus making RL research both more systematic and more accessible across the community.

Atari Games DQN Replay Dataset +3

Robust Gaussian Covariance Estimation in Nearly-Matrix Multiplication Time

no code implementations NeurIPS 2020 Jerry Li, Guanghao Ye

Previous work of Cheng et al demonstrated an algorithm that, given $N = \Omega (d^2 / \varepsilon^2)$ samples, achieved a near-optimal error of $O(\varepsilon \log 1 / \varepsilon)$, and moreover, their algorithm ran in time $\widetilde{O}(T(N, d) \log \kappa / \mathrm{poly} (\varepsilon))$, where $T(N, d)$ is the time it takes to multiply a $d \times N$ matrix by its transpose, and $\kappa$ is the condition number of $\Sigma$.

Robust Sub-Gaussian Principal Component Analysis and Width-Independent Schatten Packing

no code implementations NeurIPS 2020 Arun Jambulapati, Jerry Li, Kevin Tian

We develop two methods for the following fundamental statistical task: given an $\epsilon$-corrupted set of $n$ samples from a $d$-dimensional sub-Gaussian distribution, return an approximate top eigenvector of the covariance matrix.

An empirical investigation of the challenges of real-world reinforcement learning

1 code implementation24 Mar 2020 Gabriel Dulac-Arnold, Nir Levine, Daniel J. Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, Todd Hester

We believe that an approach that addresses our set of proposed challenges would be readily deployable in a large number of real world problems.

Continuous Control reinforcement-learning +1

Efficient Algorithms for Multidimensional Segmented Regression

1 code implementation24 Mar 2020 Ilias Diakonikolas, Jerry Li, Anastasia Voloshinov

We study the fundamental problem of fixed design {\em multidimensional segmented regression}: Given noisy samples from a function $f$, promised to be piecewise linear on an unknown set of $k$ rectangles, we want to recover $f$ up to a desired accuracy in mean-squared error.

regression

Learning Structured Distributions From Untrusted Batches: Faster and Simpler

1 code implementation NeurIPS 2020 Sitan Chen, Jerry Li, Ankur Moitra

We revisit the problem of learning from untrusted batches introduced by Qiao and Valiant [QV17].

Randomized Smoothing of All Shapes and Sizes

1 code implementation ICML 2020 Greg Yang, Tony Duan, J. Edward Hu, Hadi Salman, Ilya Razenshteyn, Jerry Li

Randomized smoothing is the current state-of-the-art defense with provable robustness against $\ell_2$ adversarial attacks.

Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments

no code implementations16 Dec 2019 Sitan Chen, Jerry Li, Zhao Song

In this paper, we give the first algorithm for learning an MLR that runs in time which is sub-exponential in $k$.

Clustering Density Estimation

Efficiently Learning Structured Distributions from Untrusted Batches

no code implementations5 Nov 2019 Sitan Chen, Jerry Li, Ankur Moitra

When $k = 1$ this is the standard robust univariate density estimation setting and it is well-understood that $\Omega (\epsilon)$ error is unavoidable.

Density Estimation

Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier Detection

1 code implementation NeurIPS 2019 Yihe Dong, Samuel B. Hopkins, Jerry Li

In robust mean estimation the goal is to estimate the mean $\mu$ of a distribution on $\mathbb{R}^d$ given $n$ independent samples, an $\varepsilon$-fraction of which have been corrupted by a malicious adversary.

Outlier Detection

Sample Efficient Toeplitz Covariance Estimation

no code implementations14 May 2019 Yonina C. Eldar, Jerry Li, Cameron Musco, Christopher Musco

In addition to results that hold for any Toeplitz $T$, we further study the important setting when $T$ is close to low-rank, which is often the case in practice.

Spectral Signatures in Backdoor Attacks

1 code implementation NeurIPS 2018 Brandon Tran, Jerry Li, Aleksander Madry

In this paper, we identify a new property of all known backdoor attacks, which we call \emph{spectral signatures}.

Data Poisoning

Twin-GAN -- Unpaired Cross-Domain Image Translation with Weight-Sharing GANs

no code implementations26 Aug 2018 Jerry Li

We present a framework for translating unlabeled images from one domain into analog images in another domain.

Translation

Privately Learning High-Dimensional Distributions

no code implementations1 May 2018 Gautam Kamath, Jerry Li, Vikrant Singhal, Jonathan Ullman

We present novel, computationally efficient, and differentially private algorithms for two fundamental high-dimensional learning problems: learning a multivariate Gaussian and learning a product distribution over the Boolean hypercube in total variation distance.

Vocal Bursts Intensity Prediction

Byzantine Stochastic Gradient Descent

no code implementations NeurIPS 2018 Dan Alistarh, Zeyuan Allen-Zhu, Jerry Li

This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of the $m$ machines which allegedly compute stochastic gradients every iteration, an $\alpha$-fraction are Byzantine, and can behave arbitrarily and adversarially.

Stochastic Optimization

Sever: A Robust Meta-Algorithm for Stochastic Optimization

1 code implementation7 Mar 2018 Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Jacob Steinhardt, Alistair Stewart

In high dimensions, most machine learning methods are brittle to even a small fraction of structured outliers.

Stochastic Optimization

Fast and Sample Near-Optimal Algorithms for Learning Multidimensional Histograms

no code implementations23 Feb 2018 Ilias Diakonikolas, Jerry Li, Ludwig Schmidt

We give an algorithm for this learning problem that uses $n = \tilde{O}_d(k/\epsilon^2)$ samples and runs in time $\tilde{O}_d(n)$.

On the limitations of first order approximation in GAN dynamics

no code implementations ICLR 2018 Jerry Li, Aleksander Madry, John Peebles, Ludwig Schmidt

This suggests that such usage of the first order approximation of the discriminator, which is a de-facto standard in all the existing GAN dynamics, might be one of the factors that makes GAN training so challenging in practice.

Communication-Efficient Distributed Learning of Discrete Distributions

no code implementations NeurIPS 2017 Ilias Diakonikolas, Elena Grigorescu, Jerry Li, Abhiram Natarajan, Krzysztof Onak, Ludwig Schmidt

For the case of structured distributions, such as k-histograms and monotone distributions, we design distributed learning algorithms that achieve significantly better communication guarantees than the naive ones, and obtain tight upper and lower bounds in several regimes.

Density Estimation

ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning

no code implementations ICML 2017 Hantian Zhang, Jerry Li, Kaan Kara, Dan Alistarh, Ji Liu, Ce Zhang

We examine training at reduced precision, both from a theoretical and practical perspective, and ask: is it possible to train models at end-to-end low precision with provable guarantees?

Quantization

On the Limitations of First-Order Approximation in GAN Dynamics

no code implementations ICML 2018 Jerry Li, Aleksander Madry, John Peebles, Ludwig Schmidt

While Generative Adversarial Networks (GANs) have demonstrated promising performance on multiple vision tasks, their learning dynamics are not yet well understood, both in theory and in practice.

The Power of Choice in Priority Scheduling

1 code implementation13 Jun 2017 Dan Alistarh, Justin Kopinsky, Jerry Li, Giorgi Nadiradze

We answer this question, showing that this strategy provides surprisingly strong guarantees: Although the single-choice process, where we always insert and remove from a single randomly chosen queue, has degrading cost, going to infinity as we increase the number of steps, in the two choice process, the expected rank of a removed element is $O( n )$ while the expected worst-case cost is $O( n \log n )$.

Data Structures and Algorithms Distributed, Parallel, and Cluster Computing

Robustly Learning a Gaussian: Getting Optimal Error, Efficiently

no code implementations12 Apr 2017 Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Ankur Moitra, Alistair Stewart

We give robust estimators that achieve estimation error $O(\varepsilon)$ in the total variation distance, which is optimal up to a universal constant that is independent of the dimension.

Being Robust (in High Dimensions) Can Be Practical

2 code implementations ICML 2017 Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Ankur Moitra, Alistair Stewart

Robust estimation is much more challenging in high dimensions than it is in one dimension: Most techniques either lead to intractable optimization problems or estimators that can tolerate only a tiny fraction of errors.

Vocal Bursts Intensity Prediction

Robust Sparse Estimation Tasks in High Dimensions

no code implementations20 Feb 2017 Jerry Li

In this paper we initiate the study of whether or not sparse estimation tasks can be performed efficiently in high dimensions, in the robust setting where an $\eps$-fraction of samples are corrupted adversarially.

Vocal Bursts Intensity Prediction

The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning

1 code implementation16 Nov 2016 Hantian Zhang, Jerry Li, Kaan Kara, Dan Alistarh, Ji Liu, Ce Zhang

When applied to linear models together with double sampling, we save up to another 1. 7x in data movement compared with uniform quantization.

Quantization

QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

2 code implementations NeurIPS 2017 Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, Milan Vojnovic

In this paper, we propose Quantized SGD (QSGD), a family of compression schemes which allow the compression of gradient updates at each node, while guaranteeing convergence under standard assumptions.

Image Classification Quantization +2

Fast Algorithms for Segmented Regression

no code implementations14 Jul 2016 Jayadev Acharya, Ilias Diakonikolas, Jerry Li, Ludwig Schmidt

We study the fixed design segmented regression problem: Given noisy samples from a piecewise linear function $f$, we want to recover $f$ up to a desired accuracy in mean-squared error.

regression

Robust Estimators in High Dimensions without the Computational Intractability

2 code implementations21 Apr 2016 Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Ankur Moitra, Alistair Stewart

We study high-dimensional distribution learning in an agnostic setting where an adversary is allowed to arbitrarily corrupt an $\varepsilon$-fraction of the samples.

Vocal Bursts Intensity Prediction

A Nearly Optimal and Agnostic Algorithm for Properly Learning a Mixture of k Gaussians, for any Constant k

no code implementations3 Jun 2015 Jerry Li, Ludwig Schmidt

One notion of learning a GMM is proper learning: here, the goal is to find a mixture of $k$ Gaussians $\mathcal{M}$ that is close to the density $f$ of the unknown distribution from which we draw samples.

Learning Theory

Sample-Optimal Density Estimation in Nearly-Linear Time

no code implementations1 Jun 2015 Jayadev Acharya, Ilias Diakonikolas, Jerry Li, Ludwig Schmidt

Let $f$ be the density function of an arbitrary univariate distribution, and suppose that $f$ is $\mathrm{OPT}$-close in $L_1$-distance to an unknown piecewise polynomial function with $t$ interval pieces and degree $d$.

Density Estimation

Lower Bounds for Exact Model Counting and Applications in Probabilistic Databases

no code implementations26 Sep 2013 Paul Beame, Jerry Li, Sudeepa Roy, Dan Suciu

The best current methods for exactly computing the number of satisfying assignments, or the satisfying probability, of Boolean formulas can be seen, either directly or indirectly, as building 'decision-DNNF' (decision decomposable negation normal form) representations of the input Boolean formulas.

Negation

Cannot find the paper you are looking for? You can Submit a new open access paper.