Browse > Methodology > Stochastic Optimization

Stochastic Optimization

69 papers with code · Methodology

State-of-the-art leaderboards

No evaluation results yet. Help compare methods by submit evaluation metrics.

Greatest papers with code

Revisiting Distributed Synchronous SGD

4 Apr 2016tensorflow/models

Distributed training of deep learning models on large-scale training data is typically conducted with asynchronous stochastic optimization to maximize the rate of updates, at the cost of additional noise introduced from asynchrony.

STOCHASTIC OPTIMIZATION

Adaptive Gradient Methods with Dynamic Bound of Learning Rate

ICLR 2019 Luolc/AdaBound

Adaptive optimization methods such as AdaGrad, RMSProp and Adam have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates.

STOCHASTIC OPTIMIZATION

SGDR: Stochastic Gradient Descent with Warm Restarts

13 Aug 2016rwightman/pytorch-image-models

Partial warm restarts are also gaining popularity in gradient-based optimization to improve the rate of convergence in accelerated gradient schemes to deal with ill-conditioned functions.

STOCHASTIC OPTIMIZATION

Greedy Step Averaging: A parameter-free stochastic optimization method

11 Nov 2016TalkingData/Fregata

In this paper we present the greedy step averaging(GSA) method, a parameter-free stochastic optimization algorithm for a variety of machine learning problems.

STOCHASTIC OPTIMIZATION

Deep learning with Elastic Averaging SGD

NeurIPS 2015 cerndb/dist-keras

We empirically demonstrate that in the deep learning setting, due to the existence of many local optima, allowing more exploration can lead to the improved performance.

IMAGE CLASSIFICATION STOCHASTIC OPTIMIZATION

Averaging Weights Leads to Wider Optima and Better Generalization

14 Mar 2018timgaripov/swa

Deep neural networks are typically trained by optimizing a loss function with an SGD variant, in conjunction with a decaying learning rate, until convergence.

IMAGE CLASSIFICATION STOCHASTIC OPTIMIZATION

DeepType: Multilingual Entity Linking by Neural Type System Evolution

3 Feb 2018openai/deeptype

The wealth of structured (e. g. Wikidata) and unstructured data about the world available today presents an incredible opportunity for tomorrow's Artificial Intelligence.

ENTITY LINKING STOCHASTIC OPTIMIZATION

Second-Order Stochastic Optimization for Machine Learning in Linear Time

12 Feb 2016darkonhub/darkon

First-order stochastic methods are the state-of-the-art in large-scale machine learning optimization owing to efficient per-iteration complexity.

STOCHASTIC OPTIMIZATION

On the insufficiency of existing momentum schemes for Stochastic Optimization

ICLR 2018 rahulkidambi/AccSGD

Extensive empirical results in this paper show that ASGD has performance gains over HB, NAG, and SGD.

STOCHASTIC OPTIMIZATION

Biased Importance Sampling for Deep Neural Network Training

31 May 2017idiap/importance-sampling

Importance sampling has been successfully used to accelerate stochastic optimization in many convex problems.

IMAGE CLASSIFICATION LANGUAGE MODELLING STOCHASTIC OPTIMIZATION