Search Results for author:
Found 26 papers, 5 papers with code
Finding Optimal Policy for Queueing Models: New Parameterization
In this work, we investigate the optimization aspects of the queueing model as a RL environment and provide insight to learn the optimal policy efficiently.
Nesterov Accelerated Shuffling Gradient Method for Convex Optimization
This rate is better than that of any other shuffling gradient methods in convex regime.
Adaptive Stochastic Optimization
Optimization lies at the heart of machine learning and signal processing.
A Novel Smoothed Loss and Penalty Function for Noncrossing Composite Quantile Estimation via Deep Neural Networks
However, one major shortcoming of composite quantile estimation in neural networks is the quantile crossover problem.
Feature Engineering and Forecasting via Derivative-free Optimization and Ensemble of Sequence-to-sequence Networks with Applications in Renewable Energy
This study introduces a framework for the forecasting, reconstruction and feature engineering of multivariate processes along with its renewable energy applications.
Linear interpolation gives better gradients than Gaussian smoothing in derivative-free optimization
We then demonstrate via rigorous analysis of the variance and by numerical comparisons on reinforcement learning tasks that the Gaussian sampling method used in [Salimans et al. 2016] is significantly inferior to the orthogonal sampling used in [Choromaski et al. 2018] as well as more general interpolation methods.
A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization
To this end, we use the results in [Berahas et al., 2019] and show how each method can satisfy the sufficient conditions, possibly only with some sufficiently large probability at each iteration, as happens to be the case with Gaussian smoothing and smoothing on a sphere.
Optimization and Control
Novel and Efficient Approximations for Zero-One Loss of Linear Classifiers
In this work, we show that in the case of linear predictors, the expected error and the expected ranking loss can be effectively approximated by smooth functions whose closed form expressions and those of their first (and second) order derivatives depend on the first and second moments of the data distribution, which can be precomputed.
Inexact SARAH Algorithm for Stochastic Optimization
We develop and analyze a variant of the SARAH algorithm, which does not require computation of the exact gradient.
New Convergence Aspects of Stochastic Gradient Algorithms
We show the convergence of SGD for strongly convex objective function without using bounded gradient assumption when $\{\eta_t\}$ is a diminishing sequence and $\sum_{t=0}^\infty \eta_t \rightarrow \infty$.
An Empirical Analysis of Constrained Support Vector Quantile Regression for Nonparametric Probabilistic Forecasting of Wind Power
Uncertainty analysis in the form of probabilistic forecasting can provide significant improvements in decision-making processes in the smart power grid for better integrating renewable energies such as wind.
SGD and Hogwild! Convergence Without the Bounded Gradients Assumption
In (Bottou et al., 2016), a new analysis of convergence of SGD is performed under the assumption that stochastic gradients are bounded with respect to the true gradient norm.
Directly and Efficiently Optimizing Prediction Error and AUC of Linear Classifiers
We show that even when data is not normally distributed, computed derivatives are sufficiently useful to render an efficient optimization method and high quality solutions.
When Does Stochastic Gradient Algorithm Work Well?
In this paper, we consider a general stochastic optimization problem which is often at the core of supervised learning, such as deep learning and linear classification.
A Stochastic Trust Region Algorithm Based on Careful Step Normalization
An algorithm is proposed for solving stochastic and finite sum minimization problems.
Smooth Pinball Neural Network for Probabilistic Forecasting of Wind Power
Multiple quantiles are estimated to form 10%, to 90% prediction intervals which are evaluated using a quantile score and reliability measures.
Optimization Methods for Supervised Machine Learning: From Linear Models to Deep Learning
We then discuss some of the distinctive features of these optimization problems, focusing on the examples of logistic regression and the training of deep neural networks.
Stochastic Recursive Gradient Algorithm for Nonconvex Optimization
In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses.
Black-Box Optimization in Machine Learning with Trust Region Based Derivative Free Algorithm
In this work, we utilize a Trust Region based Derivative Free Optimization (DFO-TR) method to directly maximize the Area Under Receiver Operating Characteristic Curve (AUC), which is a nonsmooth, noisy function.
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient
In this paper, we propose a StochAstic Recursive grAdient algoritHm (SARAH), as well as its practical variant SARAH+, as a novel approach to the finite-sum minimization problems.
Optimal Generalized Decision Trees via Integer Programming
Decision trees have been a very popular class of predictive models for decades due to their interpretability and good performance on categorical features.
Proximal Quasi-Newton Methods for Regularized Convex Optimization with Linear and Accelerated Sublinear Convergence Rates
In [19], a general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed and a sublinear global convergence rate has been established.
Practical Inexact Proximal Quasi-Newton Method with Global Complexity Analysis
Recently several methods were proposed for sparse optimization which make careful use of second-order information [10, 28, 16, 3] to improve local convergence rates.
Efficiently Using Second Order Information in Large l1 Regularization Problems
We propose a novel general algorithm LHAC that efficiently uses second-order information to train a class of large-scale l1-regularized problems.
Sparse Inverse Covariance Selection via Alternating Linearization Methods
Gaussian graphical models are of great interest in statistical learning.
Fast Alternating Linearization Methods for Minimizing the Sum of Two Convex Functions
We present in this paper first-order alternating linearization algorithms based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions.
