Search Results for author:
Found 10 papers, 0 papers with code
GPU Accelerated Sub-Sampled Newton's Method
In particular, in convex settings, we consider variants of classical Newton\textsf{'}s method in which the Hessian and/or the gradient are randomly sub-sampled.
Out-of-sample extension of graph adjacency spectral embedding
Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to apply a learned embedding to observations not seen in the initial training sample.
Invariance of Weight Distributions in Rectified MLPs
An interesting approach to analyzing neural networks that has received renewed attention is to examine the equivalent kernel of the neural network.
GIANT: Globally Improved Approximate Newton Method for Distributed Optimization
For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method.
Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study
While first-order optimization methods such as stochastic gradient descent (SGD) are popular in machine learning (ML), they come with well-known deficiencies, including relatively-slow convergence, sensitivity to the settings of hyper-parameters such as learning rate, stagnation at high training errors, and difficulty in escaping flat regions and saddle points.
Union of Intersections (UoI) for Interpretable Data Driven Discovery and Prediction
The increasing size and complexity of scientific data could dramatically enhance discovery and prediction for basic scientific applications.
Sub-sampled Newton Methods with Non-uniform Sampling
As second-order methods prove to be effective in finding the minimizer to a high-precision, in this work, we propose randomized Newton-type algorithms that exploit \textit{non-uniform} sub-sampling of $\{\nabla^2 f_i(w)\}_{i=1}^{n}$, as well as inexact updates, as means to reduce the computational complexity.
FLAG n' FLARE: Fast Linearly-Coupled Adaptive Gradient Methods
We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions.
Sub-Sampled Newton Methods I: Globally Convergent Algorithms
As a remedy, for all of our algorithms, we also give global convergence results for the case of inexact updates where such linear system is solved only approximately.
Sub-Sampled Newton Methods II: Local Convergence Rates
In such problems, sub-sampling as a way to reduce $n$ can offer great amount of computational efficiency.
