Search Results for author:
Found 12 papers, 2 papers with code
Complexity of Training ReLU Neural Networks
In this paper, we explore some basic questions on complexity of training Neural networks with ReLU activation function.
First-order methods for Stochastic Variational Inequality problems with Function Constraints
Second, to obtain the optimal operator complexity for smooth deterministic problems, we present a novel single-loop Adaptive Lagrangian Extrapolation~(\texttt{AdLagEx}) method that can adaptively search for and explicitly bound the Lagrange multipliers.
Accelerated Primal-Dual Methods for Convex-Strongly-Concave Saddle Point Problems
We investigate a primal-dual (PD) method for the saddle point problem (SPP) that uses a linear approximation of the primal function instead of the standard proximal step, resulting in a linearized PD (LPD) method.
Optimal Algorithms for Differentially Private Stochastic Monotone Variational Inequalities and Saddle-Point Problems
We show that a stochastic approximation variant of these algorithms attains risk bounds vanishing as a function of the dataset size, with respect to the strong gap function; and a sampling with replacement variant achieves optimal risk bounds with respect to a weak gap function.
A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization
We also establish new convergence complexities to achieve an approximate KKT solution when the objective can be smooth/nonsmooth, deterministic/stochastic and convex/nonconvex with complexity that is on a par with gradient descent for unconstrained optimization problems in respective cases.
Differentially Private Synthetic Mixed-Type Data Generation For Unsupervised Learning
We introduce the DP-auto-GAN framework for synthetic data generation, which combines the low dimensional representation of autoencoders with the flexibility of Generative Adversarial Networks (GANs).
Faster width-dependent algorithm for mixed packing and covering LPs
As a special case of our result, we report a $1+\eps$ approximation algorithm for the densest subgraph problem which runs in time $O(md/ \eps)$, where $m$ is the number of edges in the graph and $d$ is the maximum graph degree.
Differentially Private Mixed-Type Data Generation For Unsupervised Learning
In this work we introduce the DP-auto-GAN framework for synthetic data generation, which combines the low dimensional representation of autoencoders with the flexibility of GANs.
Stochastic First-order Methods for Convex and Nonconvex Functional Constrained Optimization
For large-scale and stochastic problems, we present a more practical proximal point method in which the approximate solutions of the subproblems are computed by the aforementioned ConEx method.
Complexity of Training ReLU Neural Network
In this paper, we explore some basic questions on the complexity of training neural networks with ReLU activation function.
Theoretical properties of the global optimizer of two-layer Neural Network
We essentially show that these non-singular hidden layer matrix satisfy a ``"good" property for these big class of activation functions.
Theoretical properties of the global optimizer of two layer neural network
We look at this problem in the setting where the number of parameters is greater than the number of sampled points.
