Search Results for author:
Found 46 papers, 28 papers with code
Practical Deep Learning with Bayesian Principles
Importantly, the benefits of Bayesian principles are preserved: predictive probabilities are well-calibrated, uncertainties on out-of-distribution data are improved, and continual-learning performance is boosted.
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
Uncertainty computation in deep learning is essential to design robust and reliable systems.
Fast Variational Learning in State-Space Gaussian Process Models
Gaussian process (GP) regression with 1D inputs can often be performed in linear time via a stochastic differential equation formulation.
Approximate Inference Turns Deep Networks into Gaussian Processes
Deep neural networks (DNN) and Gaussian processes (GP) are two powerful models with several theoretical connections relating them, but the relationship between their training methods is not well understood.
Variational Message Passing with Structured Inference Networks
Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret.
Scalable Training of Inference Networks for Gaussian-Process Models
Inference in Gaussian process (GP) models is computationally challenging for large data, and often difficult to approximate with a small number of inducing points.
Continual Deep Learning by Functional Regularisation of Memorable Past
Continually learning new skills is important for intelligent systems, yet standard deep learning methods suffer from catastrophic forgetting of the past.
Training Binary Neural Networks using the Bayesian Learning Rule
Our work provides a principled approach for training binary neural networks which justifies and extends existing approaches.
TD-Regularized Actor-Critic Methods
Actor-critic methods can achieve incredible performance on difficult reinforcement learning problems, but they are also prone to instability.
Variational Learning is Effective for Large Deep Networks
We give extensive empirical evidence against the common belief that variational learning is ineffective for large neural networks.
SAM as an Optimal Relaxation of Bayes
Sharpness-aware minimization (SAM) and related adversarial deep-learning methods can drastically improve generalization, but their underlying mechanisms are not yet fully understood.
Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
In this paper, we propose a new algorithm called Conjugate-computation Variational Inference (CVI) which brings the best of the two worlds together -- it uses conjugate computations for the conjugate terms and employs stochastic gradients for the rest.
Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations
Natural-gradient methods enable fast and simple algorithms for variational inference, but due to computational difficulties, their use is mostly limited to \emph{minimal} exponential-family (EF) approximations.
Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures
Our generalization enables us to establish a connection between Stein's lemma and the reparamterization trick to derive gradients of expectations of a large class of functions under weak assumptions.
Variational Imitation Learning with Diverse-quality Demonstrations
Learning from demonstrations can be challenging when the quality of demonstrations is diverse, and even more so when the quality is unknown and there is no additional information to estimate the quality.
Knowledge-Adaptation Priors
Humans and animals have a natural ability to quickly adapt to their surroundings, but machine-learning models, when subjected to changes, often require a complete retraining from scratch.
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient
Uncertainty estimation in large deep-learning models is a computationally challenging task, where it is difficult to form even a Gaussian approximation to the posterior distribution.
Dual Parameterization of Sparse Variational Gaussian Processes
Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits.
Memory-Based Dual Gaussian Processes for Sequential Learning
Sequential learning with Gaussian processes (GPs) is challenging when access to past data is limited, for example, in continual and active learning.
Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning
Marginal-likelihood based model-selection, even though promising, is rarely used in deep learning due to estimation difficulties.
Handling the Positive-Definite Constraint in the Bayesian Learning Rule
The Bayesian learning rule is a natural-gradient variational inference method, which not only contains many existing learning algorithms as special cases but also enables the design of new algorithms.
Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
Riemannian submanifold optimization with momentum is computationally challenging because, to ensure that the iterates remain on the submanifold, we often need to solve difficult differential equations.
The Memory Perturbation Equation: Understanding Model's Sensitivity to Data
Understanding model's sensitivity to its training data is crucial but can also be challenging and costly, especially during training.
Bayesian Nonparametric Poisson-Process Allocation for Time-Sequence Modeling
We model the intensity of each sequence as an infinite mixture of latent functions, each of which is obtained using a function drawn from a Gaussian process.
Subset-of-Data Variational Inference for Deep Gaussian-Processes Regression
Deep Gaussian Processes (DGPs) are multi-layer, flexible extensions of Gaussian processes but their training remains challenging.
Exploiting Inferential Structure in Neural Processes
Neural Processes (NPs) are appealing due to their ability to perform fast adaptation based on a context set.
Beyond Unfolding: Exact Recovery of Latent Convex Tensor Decomposition under Reshuffling
Exact recovery of tensor decomposition (TD) methods is a desirable property in both unsupervised learning and scientific data analysis.
Vprop: Variational Inference using RMSprop
Overall, this paper presents Vprop as a principled, computationally-efficient, and easy-to-implement method for Bayesian deep learning.
Variational Adaptive-Newton Method for Explorative Learning
We present the Variational Adaptive Newton (VAN) method which is a black-box optimization method especially suitable for explorative-learning tasks such as active learning and reinforcement learning.
Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions
We also give a convergence-rate analysis of our method and many other previous methods which exploit the geometry of the space.
UAVs using Bayesian Optimization to Locate WiFi Devices
We address the problem of localizing non-collaborative WiFi devices in a large region.
Fast Dual Variational Inference for Non-Conjugate LGMs
Latent Gaussian models (LGMs) are widely used in statistics and machine learning.
Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models
Bayesian inference plays an important role in advancing machine learning, but faces computational challenges when applied to complex models such as deep neural networks.
A Generalization Bound for Online Variational Inference
Our work in this paper presents theoretical justifications in favor of online algorithms relying on approximate Bayesian methods.
VILD: Variational Imitation Learning with Diverse-quality Demonstrations
However, the quality of demonstrations in reality can be diverse, since it is easier and cheaper to collect demonstrations from a mix of experts and amateurs.
Tractable structured natural gradient descent using local parameterizations
Natural-gradient descent (NGD) on structured parameter spaces (e. g., low-rank covariances) is computationally challenging due to difficult Fisher-matrix computations.
Bridging the Gap Between Target Networks and Functional Regularization
Our findings emphasize that Functional Regularization can be used as a drop-in replacement for Target Networks and result in performance improvement.
The Bayesian Learning Rule
The rule, derived from Bayesian principles, yields a wide-range of algorithms from fields such as optimization, deep learning, and graphical models.
Structured second-order methods via natural gradient descent
In this paper, we propose new structured second-order methods and structured adaptive-gradient methods obtained by performing natural-gradient descent on structured parameter spaces.
Can Calibration Improve Sample Prioritization?
Calibration can reduce overconfident predictions of deep neural networks, but can calibration also accelerate training?
Bridging the Gap Between Target Networks and Functional Regularization
However, learning the value function via bootstrapping often leads to unstable training due to fast-changing target values.
The Lie-Group Bayesian Learning Rule
This simplifies all three difficulties for many cases, providing flexible parametrizations through group's action, simple gradient computation through reparameterization, and updates that always stay on the manifold.
Variational Bayes Made Easy
Variational Bayes is a popular method for approximate inference but its derivation can be cumbersome.
Model Merging by Uncertainty-Based Gradient Matching
Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail?
Position Paper: Bayesian Deep Learning in the Age of Large-Scale AI
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets.
Conformal Prediction via Regression-as-Classification
Conformal prediction (CP) for regression can be challenging, especially when the output distribution is heteroscedastic, multimodal, or skewed.
