Search Results for author:
Found 16 papers, 0 papers with code
Non-asymptotic Analysis of Biased Stochastic Approximation Scheme
We illustrate these settings with the online EM algorithm and the policy-gradient method for average reward maximization in reinforcement learning.
An Optimistic Acceleration of AMSGrad for Nonconvex Optimization
We propose a new variant of AMSGrad, a popular adaptive gradient based optimization algorithm widely used for training deep neural networks.
On the Global Convergence of (Fast) Incremental Expectation Maximization Methods
To alleviate this problem, Neal and Hinton have proposed an incremental version of the EM (iEM) in which at each iteration the conditional expectation of the latent data (E-step) is updated only for a mini-batch of observations.
FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
Communication complexity and privacy are the two key challenges in Federated Learning where the goal is to perform a distributed learning through a large volume of devices.
HWA: Hyperparameters Weight Averaging in Bayesian Neural Networks
Bayesian neural networks attempt to combine the strong predictive performance of neural networks with formal quantification of uncertainty of the predicted output in the Bayesian framework.
Towards Better Generalization of Adaptive Gradient Methods
Adaptive gradient methods such as AdaGrad, RMSprop and Adam have been optimizers of choice for deep learning due to their fast training speed.
MISSO: Minimization by Incremental Stochastic Surrogate Optimization for Large Scale Nonconvex and Nonsmooth Problems
Many constrained, nonconvex and nonsmooth optimization problems can be tackled using the majorization-minimization (MM) method which alternates between constructing a surrogate function which upper bounds the objective function, and then minimizing this surrogate.
Convergent Adaptive Gradient Methods in Decentralized Optimization
Specifically, we propose a general algorithmic framework that can convert existing adaptive gradient methods to their decentralized counterparts.
On the Convergence of Decentralized Adaptive Gradient Methods
Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks.
Layer-wise and Dimension-wise Locally Adaptive Federated Learning
In the emerging paradigm of Federated Learning (FL), large amount of clients such as mobile devices are used to train possibly high-dimensional models on their respective data.
A Class of Two-Timescale Stochastic EM Algorithms for Nonconvex Latent Variable Models
We motivate the choice of a double dynamic by invoking the variance reduction virtue of each stage of the method on both sources of noise: the index sampling for the incremental update and the MC approximation.
Joint learning of object graph and relation graph for visual question answering
Modeling visual question answering(VQA) through scene graphs can significantly improve the reasoning accuracy and interpretability.
On Distributed Adaptive Optimization with Gradient Compression
We study COMP-AMS, a distributed optimization framework based on gradient averaging and adaptive AMSGrad algorithm.
Variational Flow Graphical Model
VFGs learn the representation of high dimensional data via a message-passing scheme by integrating flow-based functions through variational inference.
FeatureBox: Feature Engineering on GPUs for Massive-Scale Ads Systems
In this paper, we propose FeatureBox, a novel end-to-end training framework that pipelines the feature extraction and the training on GPU servers to save the intermediate I/O of the feature extraction.
STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models
We propose in this paper, STANLEY, a STochastic gradient ANisotropic LangEvin dYnamics, for sampling high dimensional data.
