Search Results for author:
Found 34 papers, 13 papers with code
Variational Inference for Sequential Data with Future Likelihood Estimates
The estimated future likelihoods form the core of our new low-variance gradient estimator.
An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network
The recent theoretical analysis of deep neural networks in their infinite-width limits has deepened our understanding of initialisation, feature learning, and training of those networks, and brought new practical techniques for finding appropriate hyperparameters, learning network weights, and performing inference.
Learning Symmetrization for Equivariance with Orbit Distance Minimization
We present a general framework for symmetrizing an arbitrary neural-network architecture and making it equivariant with respect to a given group.
Regularizing Towards Soft Equivariance Under Mixed Symmetries
The key component of our method is what we call equivariance regularizer for a given type of symmetries, which measures how much a model is equivariant with respect to the symmetries of the type.
Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning
We consider the optimisation of large and shallow neural networks via gradient flow, where the output of each hidden node is scaled by some positive parameter.
Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference
We present a static analysis for discovering differentiable or more generally smooth parts of a given probabilistic program, and show how the analysis can be used to improve the pathwise gradient estimator, one of the most popular methods for posterior inference and model learning.
Learning Symmetric Rules with SATNet
SATNet is a differentiable constraint solver with a custom backpropagation algorithm, which can be used as a layer in a deep-learning system.
Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility
Under this model, we show that each layer of the infinite-width neural network can be characterised by two simple quantities: a non-negative scalar parameter and a L\'evy measure on the positive reals.
LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimation
We consider the problem of learning from observation (LfO), in which the agent aims to mimic the expert's behavior from the state-only demonstrations by experts.
DemoDICE: Offline Imitation Learning with Supplementary Imperfect Demonstrations
We consider offline imitation learning (IL), which aims to mimic the expert's behavior from its demonstration without further interaction with the environment.
Scale Mixtures of Neural Network Gaussian Processes
We show that simply introducing a scale prior on the last-layer parameters can turn infinitely-wide neural networks of any architecture into a richer class of stochastic processes.
$α$-Stable convergence of heavy-tailed infinitely-wide neural networks
We consider infinitely-wide multi-layer perceptrons (MLPs) which are limits of standard deep feed-forward neural networks.
Meta-Learning an Inference Algorithm for Probabilistic Programs
The parameters of the networks are learnt from a training set by our meta-algorithm.
Adaptive Strategy for Resetting a Non-stationary Markov Chain during Learning via Joint Stochastic Approximation
The non-stationary kernel problem refers to the degraded performance of the algorithm due to the constant change of the transition kernel of the chain throughout the run of the algorithm.
Bayesian Policy Search for Stochastic Domains
In this work, we cast policy search in stochastic domains as a Bayesian inference problem and provide a scheme for encoding such problems as nested probabilistic programs.
Probabilistic Programs with Stochastic Conditioning
We tackle the problem of conditioning probabilistic programs on distributions of observable variables.
On Correctness of Automatic Differentiation for Non-Differentiable Functions
For these PAP functions, we propose a new type of derivatives, called intensional derivatives, and prove that these derivatives always exist and coincide with standard derivatives for almost all inputs.
Stochastically Differentiable Probabilistic Programs
Probabilistic programs with mixed support (both continuous and discrete latent random variables) commonly appear in many probabilistic programming systems (PPSs).
Differentiable Algorithm for Marginalising Changepoints
We present an algorithm for marginalising changepoints in time-series models that assume a fixed number of unknown changepoints.
Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support
Universal probabilistic programming systems (PPSs) provide a powerful framework for specifying rich probabilistic models.
Towards Verified Stochastic Variational Inference for Probabilistic Programs
In this paper, we analyse one of the most fundamental and versatile variational inference algorithms, called score estimator, using tools from denotational semantics and program analysis.
LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables.
An Introduction to Probabilistic Programming
We start with a discussion of model-based reasoning and explain why conditioning is a foundational computation central to the fields of probabilistic machine learning and artificial intelligence.
Inference Trees: Adaptive Inference with Exploration
We introduce inference trees (ITs), a new class of inference methods that build on ideas from Monte Carlo tree search to perform adaptive sampling in a manner that balances exploration with exploitation, ensures consistency, and alleviates pathologies in existing adaptive methods.
Reparameterization Gradient for Non-differentiable Models
We tackle the challenge by generalizing the reparameterization trick, one of the most effective techniques for addressing the variance issue for differentiable models, so that the trick works for non-differentiable models as well.
Hamiltonian Monte Carlo for Probabilistic Programs with Discontinuities
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs).
Towards a Testable Notion of Generalization for Generative Adversarial Networks
We consider the question of how to assess generative adversarial networks, in particular with respect to whether or not they generalise beyond memorising the training data.
On Nesting Monte Carlo Estimators
Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation.
A Convenient Category for Higher-Order Probability Theory
Higher-order probabilistic programming languages allow programmers to write sophisticated models in machine learning and statistics in a succinct and structured way, but step outside the standard measure-theoretic formalization of probability theory.
On the Pitfalls of Nested Monte Carlo
In this paper, we analyse the behaviour of nested Monte Carlo (NMC) schemes, for which classical convergence proofs are insufficient.
Spreadsheet Probabilistic Programming
We demonstrate this by developing a native Excel implementation of both a particle Markov Chain Monte Carlo variant and black-box variational inference for spreadsheet probabilistic programming.
Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints
We study the semantic foundation of expressive probabilistic programming languages, that support higher-order functions, continuous distributions, and soft constraints (such as Anglican, Church, and Venture).
Abstraction Refinement Guided by a Learnt Probabilistic Model
Our approach applies to parametric static analyses implemented in Datalog, and is based on counterexample-guided abstraction refinement.
Programming Languages Software Engineering D.2.4
Particle Gibbs with Ancestor Sampling for Probabilistic Programs
Particle Markov chain Monte Carlo techniques rank among current state-of-the-art methods for probabilistic program inference.
