Search Results for author:
Found 28 papers, 14 papers with code
Restricted Boltzmann Machines for Collaborative Filtering
Most of the existing approaches to collaborative filtering cannot handle very large data sets.
A Scalable Hierarchical Distributed Language Model
Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models.
Learning Label Trees for Probabilistic Modelling of Implicit Feedback
User preferences for items can be inferred from either explicit feedback, such as item ratings, or implicit feedback, such as rental histories.
Deep AutoRegressive Networks
We introduce a deep, generative autoencoder capable of learning hierarchies of distributed representations from data.
Learning word embeddings efficiently with noise-contrastive estimation
Continuous-valued word embeddings learned by neural language models have recently been shown to capture semantic and syntactic information about words very well, setting performance records on several word similarity tasks.
Neural Variational Inference and Learning in Belief Networks
Highly expressive directed latent variable models, such as sigmoid belief networks, are difficult to train on large datasets because exact inference in them is intractable and none of the approximate inference methods that have been applied to them scale well.
MuProp: Unbiased Backpropagation for Stochastic Neural Networks
Deep neural networks are powerful parametric models that can be trained efficiently using the backpropagation algorithm.
Variational inference for Monte Carlo objectives
Recent progress in deep latent variable models has largely been driven by the development of flexible and scalable variational inference methods.
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
The essence of the trick is to refactor each stochastic node into a differentiable function of its parameters and a random variable with fixed distribution.
Particle Value Functions
The policy gradients of the expected return objective can react slowly to rare rewards.
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
Learning in models with discrete latent variables is challenging due to high variance gradient estimators.
Filtering Variational Objectives
When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results.
Variational Memory Addressing in Generative Models
Aiming to augment generative models with external memory, we interpret the output of a memory module with stochastic addressing as a conditional mixture distribution, where a read operation corresponds to sampling a discrete memory address and retrieving the corresponding content from memory.
Disentangling by Factorising
We define and address the problem of unsupervised learning of disentangled representations on data generated from independent factors of variation.
Implicit Reparameterization Gradients
By providing a simple and efficient way of computing low-variance gradients of continuous random variables, the reparameterization trick has become the technique of choice for training a variety of latent variable models.
Resampled Priors for Variational Autoencoders
We propose Learned Accept/Reject Sampling (LARS), a method for constructing richer priors using rejection sampling with a learned acceptance function.
Attentive Neural Processes
Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions.
Learning to Control Visual Abstractions for Structured Exploration in Deep Reinforcement Learning
Exploration in environments with sparse rewards is a key challenge for reinforcement learning.
Monte Carlo Gradient Estimation in Machine Learning
This paper is a broad and accessible survey of the methods we have at our disposal for Monte Carlo gradient estimation in machine learning and across the statistical sciences: the problem of computing the gradient of an expectation of a function with respect to parameters defining the distribution that is integrated; the problem of sensitivity analysis.
Sparse Orthogonal Variational Inference for Gaussian Processes
We introduce a new interpretation of sparse variational approximations for Gaussian processes using inducing points, which can lead to more scalable algorithms than previous methods.
Q-Learning in enormous action spaces via amortized approximate maximization
Applying Q-learning to high-dimensional or continuous action spaces can be difficult due to the required maximization over the set of possible actions.
The Lipschitz Constant of Self-Attention
Lipschitz constants of neural networks have been explored in various contexts in deep learning, such as provable adversarial robustness, estimating Wasserstein distance, stabilising training of GANs, and formulating invertible neural networks.
DisARM: An Antithetic Gradient Estimator for Binary Latent Variables
Applying antithetic sampling over the augmenting variables yields a relatively low-variance and unbiased estimator applicable to any model with binary latent variables.
Generalized Doubly-Reparameterized Gradient Estimators
Efficient low-variance gradient estimation enabled by the reparameterization trick (RT) has been essential to the success of variational autoencoders.
Generalized Doubly Reparameterized Gradient Estimators
Efficient low-variance gradient estimation enabled by the reparameterization trick (RT) has been essential to the success of variational autoencoders.
Coupled Gradient Estimators for Discrete Latent Variables
Training models with discrete latent variables is challenging due to the high variance of unbiased gradient estimators.
Unbiased Gradient Estimation with Balanced Assignments for Mixtures of Experts
Training large-scale mixture of experts models efficiently on modern hardware requires assigning datapoints in a batch to different experts, each with a limited capacity.
Compositional Score Modeling for Simulation-based Inference
Neural Posterior Estimation methods for simulation-based inference can be ill-suited for dealing with posterior distributions obtained by conditioning on multiple observations, as they tend to require a large number of simulator calls to learn accurate approximations.
