Search Results for author:
Found 16 papers, 1 papers with code
Transfer Learning Beyond Bounded Density Ratios
We also provide a discrete analogue of our transfer inequality on the Boolean Hypercube $\{-1, 1\}^n$, and study its connections with the recent problem of Generalization on the Unseen of Abbe, Bengio, Lotfi and Rizk (ICML, 2023).
Statistical and Computational Phase Transitions in Group Testing
For the Bernoulli design, we determine the precise number of tests required to solve the associated detection problem (where the goal is to distinguish between a group testing instance and pure noise), improving both the upper and lower bounds of Truong, Aldridge, and Scarlett (2020).
The Franz-Parisi Criterion and Computational Trade-offs in High Dimensional Statistics
We define a free-energy based criterion for hardness and formally connect it to the well-established notion of low-degree hardness for a broad class of statistical problems, namely all Gaussian additive models and certain models with a sparse planted signal.
Lattice-Based Methods Surpass Sum-of-Squares in Clustering
Prior work on many similar inference tasks portends that such lower bounds strongly suggest the presence of an inherent statistical-to-computational gap for clustering, that is, a parameter regime where the clustering task is statistically possible but no polynomial-time algorithm succeeds.
On the Cryptographic Hardness of Learning Single Periodic Neurons
More precisely, our reduction shows that any polynomial-time algorithm (not necessarily gradient-based) for learning such functions under small noise implies a polynomial-time quantum algorithm for solving worst-case lattice problems, whose hardness form the foundation of lattice-based cryptography.
Self-Regularity of Non-Negative Output Weights for Overparameterized Two-Layer Neural Networks
Using a simple covering number argument, we establish that under quite mild distributional assumptions on the input/label pairs; any such network achieving a small training error on polynomially many data necessarily has a well-controlled outer norm.
It was "all" for "nothing": sharp phase transitions for noiseless discrete channels
We establish a phase transition known as the "all-or-nothing" phenomenon for noiseless discrete channels.
Statistics Theory Information Theory Information Theory Probability Statistics Theory
Free Energy Wells and Overlap Gap Property in Sparse PCA
We study a variant of the sparse PCA (principal component analysis) problem in the "hard" regime, where the inference task is possible yet no polynomial-time algorithm is known to exist.
Neural Networks and Polynomial Regression. Demystifying the Overparametrization Phenomena
Thus a message implied by our results is that parametrizing wide neural networks by the number of hidden nodes is misleading, and a more fitting measure of parametrization complexity is the number of regression coefficients associated with tensorized data.
Stationary Points of Shallow Neural Networks with Quadratic Activation Function
Next, we show that initializing below this barrier is in fact easily achieved when the weights are randomly generated under relatively weak assumptions.
Inference in High-Dimensional Linear Regression via Lattice Basis Reduction and Integer Relation Detection
Using a novel combination of the PSLQ integer relation detection, and LLL lattice basis reduction algorithms, we propose a polynomial-time algorithm which provably recovers a $\beta^*\in\mathbb{R}^p$ enjoying the mixed-support assumption, from its linear measurements $Y=X\beta^*\in\mathbb{R}^n$ for a large class of distributions for the random entries of $X$, even with one measurement $(n=1)$.
The Landscape of the Planted Clique Problem: Dense subgraphs and the Overlap Gap Property
Using the first moment method, we study the densest subgraph problems for subgraphs with fixed, but arbitrary, overlap size with the planted clique, and provide evidence of a phase transition for the presence of Overlap Gap Property (OGP) at $k=\Theta\left(\sqrt{n}\right)$.
High Dimensional Linear Regression using Lattice Basis Reduction
We consider a high dimensional linear regression problem where the goal is to efficiently recover an unknown vector $\beta^*$ from $n$ noisy linear observations $Y=X\beta^*+W \in \mathbb{R}^n$, for known $X \in \mathbb{R}^{n \times p}$ and unknown $W \in \mathbb{R}^n$.
Sparse High-Dimensional Linear Regression. Algorithmic Barriers and a Local Search Algorithm
The presence of such an Overlap Gap Property phase transition, which originates in statistical physics, is known to provide evidence of an algorithmic hardness.
Orthogonal Machine Learning: Power and Limitations
Double machine learning provides $\sqrt{n}$-consistent estimates of parameters of interest even when high-dimensional or nonparametric nuisance parameters are estimated at an $n^{-1/4}$ rate.
High-Dimensional Regression with Binary Coefficients. Estimating Squared Error and a Phase Transition
c) We establish a certain Overlap Gap Property (OGP) on the space of all binary vectors \beta when n\le ck\log p for sufficiently small constant c. We conjecture that OGP is the source of algorithmic hardness of solving the minimization problem \min_{\beta}\|Y-X\beta\|_{2} in the regime n<n_{\text{LASSO/CS}}.
