Search Results for author:
Found 11 papers, 3 papers with code
Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Models
Unsupervised pre-training and transfer learning are commonly used techniques to initialize training algorithms for neural networks, particularly in settings with limited labeled data.
Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions
It is known that under general conditions, when $\mathbf{x}$ is trained by stochastic gradient descent, the evolution of these same summary statistics along training converges to the solution of an autonomous system of ODEs, called the effective dynamics.
High-dimensional SGD aligns with emerging outlier eigenspaces
We rigorously study the joint evolution of training dynamics via stochastic gradient descent (SGD) and the spectra of empirical Hessian and gradient matrices.
Optimality of Message-Passing Architectures for Sparse Graphs
We study the node classification problem on feature-decorated graphs in the sparse setting, i. e., when the expected degree of a node is $O(1)$ in the number of nodes, in the fixed-dimensional asymptotic regime, i. e., the dimension of the feature data is fixed while the number of nodes is large.
Differentially private multivariate medians
We develop novel finite-sample performance guarantees for differentially private multivariate depth-based medians, which are essentially sharp.
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling
We prove limit theorems for the trajectories of summary statistics (i. e., finite-dimensional functions) of SGD as the dimension goes to infinity.
Effects of Graph Convolutions in Multi-layer Networks
Graph Convolutional Networks (GCNs) are one of the most popular architectures that are used to solve classification problems accompanied by graphical information.
Graph Attention Retrospective
They were introduced to allow a node to aggregate information from features of neighbor nodes in a non-uniform way, in contrast to simple graph convolution which does not distinguish the neighbors of a node.
Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization
Recently there has been increased interest in semi-supervised classification in the presence of graphical information.
General Classification
Out-of-Distribution Generalization
+1
Hardness of Random Optimization Problems for Boolean Circuits, Low-Degree Polynomials, and Langevin Dynamics
For the case of Boolean circuits, our results improve the state-of-the-art bounds known in circuit complexity theory (although we consider the search problem as opposed to the decision problem).
Online stochastic gradient descent on non-convex losses from high-dimensional inference
Here one produces an estimator of an unknown parameter from independent samples of data by iteratively optimizing a loss function.
