Search Results for author:
Found 30 papers, 15 papers with code
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates.
fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measurements has the potential to reduce medical costs, minimize stress to patients and make MRI possible in applications where it is currently prohibitively slow or expensive.
GrappaNet: Combining Parallel Imaging with Deep Learning for Multi-Coil MRI Reconstruction
In this paper, we present a novel method to integrate traditional parallel imaging methods into deep neural networks that is able to generate high quality reconstructions even for high acceleration factors.
Offset Sampling Improves Deep Learning based Accelerated MRI Reconstructions by Exploiting Symmetry
Deep learning approaches to accelerated MRI take a matrix of sampled Fourier-space lines as input and produce a spatial image as output.
Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge
Conclusion: The challenge led to new developments in machine learning for image reconstruction, provided insight into the current state of the art in the field, and highlighted remaining hurdles for clinical adoption.
MRI Banding Removal via Adversarial Training
MRI images reconstructed from sub-sampled Cartesian data using deep learning techniques often show a characteristic banding (sometimes described as streaking), which is particularly strong in low signal-to-noise regions of the reconstructed image.
End-to-End Variational Networks for Accelerated MRI Reconstruction
The slow acquisition speed of magnetic resonance imaging (MRI) has led to the development of two complementary methods: acquiring multiple views of the anatomy simultaneously (parallel imaging) and acquiring fewer samples than necessary for traditional signal processing methods (compressed sensing).
Ranked #1 on
MRI Reconstruction
on fastMRI Knee 4x
Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization
Momentum methods are now used pervasively within the machine learning community for training non-convex models such as deep neural networks.
Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization
We introduce MADGRAD, a novel optimization method in the family of AdaGrad adaptive gradient methods.
Learning-Rate-Free Learning by D-Adaptation
D-Adaptation is an approach to automatically setting the learning rate which asymptotically achieves the optimal rate of convergence for minimizing convex Lipschitz functions, with no back-tracking or line searches, and no additional function value or gradient evaluations per step.
Prodigy: An Expeditiously Adaptive Parameter-Free Learner
We consider the problem of estimating the learning rate in adaptive methods, such as AdaGrad and Adam.
A Simple Practical Accelerated Method for Finite Sums
We describe a novel optimization method for finite sums (such as empirical risk minimization problems) building on the recently introduced SAGA method.
When, Why and How Much? Adaptive Learning Rate Scheduling by Refinement
To go beyond this worst-case analysis, we use the observed gradient norms to derive schedules refined for any particular task.
On the Ineffectiveness of Variance Reduced Optimization for Deep Learning
The applicability of these techniques to the hard non-convex optimization problems encountered during training of modern deep neural networks is an open problem.
MoMo: Momentum Models for Adaptive Learning Rates
MoMo uses momentum estimates of the batch losses and gradients sampled at each iteration to build a model of the loss function.
New Optimisation Methods for Machine Learning
For problems where the structure is known but the parameters unknown, we introduce an approximate maximum likelihood learning algorithm that is capable of learning a useful subclass of Gaussian graphical models.
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs).
A Comparison of learning algorithms on the Arcade Learning Environment
Reinforcement learning agents have traditionally been evaluated on small toy problems.
Controlling Covariate Shift using Balanced Normalization of Weights
We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs.
On the Curved Geometry of Accelerated Optimization
In this work we propose a differential geometric motivation for Nesterov's accelerated gradient method (AGM) for strongly-convex problems.
A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation
We consider the case where the structure of the graph to be reconstructed is known to be scale-free.
CONTROLLING COVARIATE SHIFT USING EQUILIBRIUM NORMALIZATION OF WEIGHTS
We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs.
Beyond Folklore: A Scaling Calculus for the Design and Initialization of ReLU Networks
We propose a system for calculating a "scaling constant" for layers and weights of neural networks.
Scaling Laws for the Principled Design, Initialization, and Preconditioning of ReLU Networks
Abstract In this work, we describe a set of rules for the design and initialization of well-conditioned neural networks, guided by the goal of naturally balancing the diagonal blocks of the Hessian at the start of training.
The Power of Factorial Powers: New Parameter settings for (Stochastic) Optimization
The convergence rates for convex and non-convex optimization methods depend on the choice of a host of constants, including step sizes, Lyapunov function constants and momentum constants.
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball
We show that these results still hold when using stochastic line search and stochastic Polyak stepsizes, thereby giving the first proof of convergence of these methods in the non-overparametrized regime.
Dual Averaging is Surprisingly Effective for Deep Learning Optimization
First-order stochastic optimization methods are currently the most widely used class of methods for training deep neural networks.
Stochastic Polyak Stepsize with a Moving Target
MOTAPS can be seen as a variant of the Stochastic Polyak (SP) which is also a method that also uses loss values to adjust the stepsize.
Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method
The classical AdaGrad method adapts the learning rate by dividing by the square root of a sum of squared gradients.
Directional Smoothness and Gradient Methods: Convergence and Adaptivity
We develop new sub-optimality bounds for gradient descent (GD) that depend on the conditioning of the objective along the path of optimization, rather than on global, worst-case constants.
