Search Results for author:
Found 17 papers, 5 papers with code
Escaping Saddles with Stochastic Gradients
We analyze the variance of stochastic gradients along negative curvature directions in certain non-convex machine learning models and show that stochastic gradients exhibit a strong component along these directions.
Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization
Normalization techniques such as Batch Normalization have been applied successfully for training deep neural networks.
Adaptive norms for deep learning with regularized Newton methods
We investigate the use of regularized Newton methods with adaptive norms for optimizing neural networks.
The Role of Memory in Stochastic Optimization
We first derive a general continuous-time model that can incorporate arbitrary types of memory, for both deterministic and stochastic settings.
Ellipsoidal Trust Region Methods for Neural Network Training
We investigate the use of ellipsoidal trust region constraints for second-order optimization of neural networks.
A Sub-sampled Tensor Method for Non-convex Optimization
We present a stochastic optimization method that uses a fourth-order regularized model to find local minima of smooth and potentially non-convex objective functions with a finite-sum structure.
Batch Normalization Provably Avoids Rank Collapse for Randomly Initialised Deep Networks
Randomly initialized neural networks are known to become harder to train with increasing depth, unless architectural enhancements like residual connections and batch normalization are used.
Two-Level K-FAC Preconditioning for Deep Learning
In the context of deep learning, many optimization methods use gradient covariance information in order to accelerate the convergence of Stochastic Gradient Descent.
Batch normalization provably avoids ranks collapse for randomly initialised deep networks
Randomly initialized neural networks are known to become harder to train with increasing depth, unless architectural enhancements like residual connections and batch normalization are used.
Learning Generative Models of Textured 3D Meshes from Real-World Images
Recent advances in differentiable rendering have sparked an interest in learning generative models of textured 3D meshes from image collections.
This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks
Deep neural networks that yield human interpretable decisions by architectural design have lately become an increasingly popular alternative to post hoc interpretation of traditional black-box models.
Vanishing Curvature and the Power of Adaptive Methods in Randomly Initialized Deep Networks
This paper revisits the so-called vanishing gradient phenomenon, which commonly occurs in deep randomly initialized neural networks.
Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spaces
Multi-agent control problems constitute an interesting area of application for deep reinforcement learning models with continuous action spaces.
Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework
Speech Neuroprostheses have the potential to enable communication for people with dysarthria or anarthria.
Cache Me if You Can: Accelerating Diffusion Models through Block Caching
However, one of the major drawbacks of diffusion models is that the image generation process is costly.
Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models
Our findings provide insights into the efficiency of the conditional denoising process that contribute to more practical and swift deployment of text-conditioned diffusion models.
fMPI: Fast Novel View Synthesis in the Wild with Layered Scene Representations
In this study, we propose two novel input processing paradigms for novel view synthesis (NVS) methods based on layered scene representations that significantly improve their runtime without compromising quality.
