Search Results for author:
Found 12 papers, 7 papers with code
Almost-sure convergence of iterates and multipliers in stochastic sequential quadratic optimization
In this paper, new almost-sure convergence guarantees for the primal iterates, Lagrange multipliers, and stationarity measures generated by a stochastic SQP algorithm in this subclass of methods are proved.
A Stochastic-Gradient-based Interior-Point Algorithm for Solving Smooth Bound-Constrained Optimization Problems
A stochastic-gradient-based interior-point algorithm for minimizing a continuously differentiable objective function (that may be nonconvex) subject to bound constraints is presented, analyzed, and demonstrated through experimental results.
A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear Equality Constrained Optimization with Rank-Deficient Jacobians
A sequential quadratic optimization algorithm is proposed for solving smooth nonlinear equality constrained optimization problems in which the objective function is defined by an expectation of a stochastic function.
A Subspace Acceleration Method for Minimization Involving a Group Sparsity-Inducing Regularizer
We consider the problem of minimizing an objective function that is the sum of a convex function and a group sparsity-inducing regularizer.
Optimization and Control 49M37, 65K05, 65K10, 65Y20, 68Q25, 90C30, 90C60
Sequential Quadratic Optimization for Nonlinear Equality Constrained Stochastic Optimization
It is assumed in this setting that it is intractable to compute objective function and derivative values explicitly, although one can compute stochastic function and gradient estimates.
Gradient Sampling Methods with Inexact Subproblem Solutions and Gradient Aggregation
In this paper, a strategy is proposed that allows the use of inexact solutions of these subproblems, which, as proved in the paper, can be incorporated without the loss of theoretical convergence guarantees.
Optimization and Control
Adaptive Stochastic Optimization
Optimization lies at the heart of machine learning and signal processing.
A Stochastic Trust Region Algorithm Based on Careful Step Normalization
An algorithm is proposed for solving stochastic and finite sum minimization problems.
An Accelerated Communication-Efficient Primal-Dual Optimization Framework for Structured Machine Learning
In this paper, an accelerated variant of CoCoA+ is proposed and shown to possess a convergence rate of $\mathcal{O}(1/t^2)$ in terms of reducing suboptimality.
Optimization Methods for Supervised Machine Learning: From Linear Models to Deep Learning
We then discuss some of the distinctive features of these optimization problems, focusing on the examples of logistic regression and the training of deep neural networks.
Optimization Methods for Large-Scale Machine Learning
This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications.
Primal-Dual Active-Set Methods for Isotonic Regression and Trend Filtering
Isotonic regression (IR) is a non-parametric calibration method used in supervised learning.
