Search Results for author:
Found 12 papers, 8 papers with code
Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
Adaptive gradient optimizers like Adam(W) are the default training algorithms for many deep learning architectures, such as transformers.
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
Second-order methods such as KFAC can be useful for neural net training.
Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
In this work, we identify two different settings of linear weight-sharing layers which motivate two flavours of K-FAC -- $\textit{expand}$ and $\textit{reduce}$.
Benchmarking Neural Network Training Algorithms
In order to address these challenges, we introduce a new, competitive, time-to-result benchmark using multiple workloads running on fixed hardware, the AlgoPerf: Training Algorithms benchmark.
Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization
The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks.
Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEs
Neural operators are a type of deep architecture that learns to solve (i. e. learns the nonlinear solution operator of) partial differential equations (PDEs).
Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks
We show that the resulting posterior approximation is competitive with even the gold-standard full-batch Hamiltonian Monte Carlo.
Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning
Deep neural networks are prone to overconfident predictions on outliers.
Laplace Redux -- Effortless Bayesian Deep Learning
Bayesian formulations of deep learning have been shown to have compelling theoretical properties and offer practical functional benefits, such as improved predictive uncertainty quantification and model selection.
Laplace Redux - Effortless Bayesian Deep Learning
Bayesian formulations of deep learning have been shown to have compelling theoretical properties and offer practical functional benefits, such as improved predictive uncertainty quantification and model selection.
Continual Deep Learning by Functional Regularisation of Memorable Past
Continually learning new skills is important for intelligent systems, yet standard deep learning methods suffer from catastrophic forgetting of the past.
Practical Deep Learning with Bayesian Principles
Importantly, the benefits of Bayesian principles are preserved: predictive probabilities are well-calibrated, uncertainties on out-of-distribution data are improved, and continual-learning performance is boosted.
