Search Results for author:
Found 56 papers, 36 papers with code
Rethinking Variational Inference for Probabilistic Programs with Stochastic Support
We introduce Support Decomposition Variational Inference (SDVI), a new variational inference (VI) approach for probabilistic programs with stochastic support.
Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support
The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path.
In-Context Learning Learns Label Relationships but Is Not Conventional Learning
The predictions of Large Language Models (LLMs) on downstream tasks often improve significantly when including examples of the input--label relationship in the context.
On the Expected Size of Conformal Prediction Sets
While conformal predictors reap the benefits of rigorous statistical guarantees for their error frequency, the size of their corresponding prediction sets is critical to their practical utility.
Prediction-Oriented Bayesian Active Learning
Information-theoretic approaches to active learning have traditionally focused on maximising the information gathered about the model parameters, most commonly by optimising the BALD score.
Incorporating Unlabelled Data into Bayesian Neural Networks
Conventional Bayesian Neural Networks (BNNs) cannot leverage unlabelled data to improve their predictions.
Modern Bayesian Experimental Design
Bayesian experimental design (BED) provides a powerful and general framework for optimizing the design of experiments.
CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Design
We formalize the problem of contextual optimization through the lens of Bayesian experimental design and propose CO-BED -- a general, model-agnostic framework for designing contextual experiments using information-theoretic principles.
Do Bayesian Neural Networks Need To Be Fully Stochastic?
We investigate the benefit of treating all the parameters in a Bayesian neural network stochastically and find compelling theoretical and empirical evidence that this standard construction may be unnecessary.
Learning Instance-Specific Augmentations by Capturing Local Invariances
We introduce InstaAug, a method for automatically learning input-specific augmentations from data.
A Continuous Time Framework for Discrete Denoising Models
We provide the first complete continuous time framework for denoising diffusion models of discrete data.
Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation.
Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods
We introduce implicit Deep Adaptive Design (iDAD), a new method for performing adaptive experiments in real-time with implicit models.
Online Variational Filtering and Parameter Learning
We present a variational method for online state estimation and parameter learning in state-space models (SSMs), a ubiquitous class of latent variable models for sequential data.
On Incorporating Inductive Biases into VAEs
InteL-VAEs use an intermediary set of latent variables to control the stochasticity of the encoding process, before mapping these in turn to the latent representation using a parametric function that encapsulates our desired inductive bias(es).
Learning Multimodal VAEs through Mutual Supervision
Here we introduce a novel alternative, the MEME, that avoids such explicit combinations by repurposing semi-supervised VAEs to combine information between modalities implicitly through mutual supervision.
Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
Expanding on MacKay (1992), we argue that conventional model-based methods for active learning - like BALD - have a fundamental shortfall: they fail to directly account for the test-time distribution of the input variables.
Group Equivariant Subsampling
We use these layers to construct group equivariant autoencoders (GAEs) that allow us to learn low-dimensional equivariant representations.
Expectation Programming: Adapting Probabilistic Programming Systems to Estimate Expectations Efficiently
We show that the standard computational pipeline of probabilistic programming systems (PPSs) can be inefficient for estimating expectations and introduce the concept of expectation programming to address this.
Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning
We challenge a common assumption underlying most supervised deep learning: that a model makes a prediction depending only on its parameters and the features of a single input.
Active Testing: Sample-Efficient Model Evaluation
While approaches like active learning reduce the number of labels needed for model training, existing literature largely ignores the cost of labeling test data, typically unrealistically assuming large test sets for model evaluation.
Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design
We introduce Deep Adaptive Design (DAD), a method for amortizing the cost of adaptive Bayesian experimental design that allows experiments to be run in real-time.
Certifiably Robust Variational Autoencoders
We introduce an approach for training Variational Autoencoders (VAEs) that are certifiably robust to adversarial attack.
On Statistical Bias In Active Learning: How and When To Fix It
Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution.
On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes
We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) issues.
Improving Transformation Invariance in Contrastive Representation Learning
We propose methods to strengthen the invariance properties of representations obtained by contrastive learning.
Probabilistic Programs with Stochastic Conditioning
We tackle the problem of conditioning probabilistic programs on distributions of observable variables.
Towards a Theoretical Understanding of the Robustness of Variational Autoencoders
We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations.
Capturing Label Characteristics in VAEs
We present a principled approach to incorporating labels in VAEs that captures the rich characteristic information associated with those labels.
Statistically Robust Neural Network Classification
The SRR provides a distinct and complementary measure of robust performance, compared to natural and adversarial risk.
A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments
We introduce a fully stochastic gradient based approach to Bayesian optimal experimental design (BOED).
Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support
Universal probabilistic programming systems (PPSs) provide a powerful framework for specifying rich probabilistic models.
Amortized Rejection Sampling in Universal Probabilistic Programming
Naive approaches to amortized inference in probabilistic programs with unbounded loops can produce estimators with infinite variance.
Efficient Bayesian Inference for Nested Simulators
We introduce two approaches for conducting efficient Bayesian inference in stochastic simulators containing nested stochastic sub-procedures, i. e., internal procedures for which the density cannot be calculated directly such as rejection sampling loops.
Amortized Monte Carlo Integration
At runtime, samples are produced separately from each amortized proposal, before being combined to an overall estimate of the expectation.
Improving VAEs' Robustness to Adversarial Attack
We make significant advances in addressing this issue by introducing methods for producing adversarially robust VAEs.
On the Fairness of Disentangled Representations
Recently there has been a significant interest in learning disentangled representations, as they promise increased interpretability, generalization to unseen scenarios and faster learning on downstream tasks.
Hijacking Malaria Simulators with Probabilistic Programming
Epidemiology simulations have become a fundamental tool in the fight against the epidemics of various infectious diseases like AIDS and malaria.
Variational Bayesian Optimal Experimental Design
Bayesian optimal experimental design (BOED) is a principled framework for making efficient use of limited experimental resources.
LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables.
Disentangling Disentanglement in Variational Autoencoders
We develop a generalisation of disentanglement in VAEs---decomposition of the latent representation---characterising it as the fulfilment of two factors: a) the latent encodings of the data having an appropriate level of overlap, and b) the aggregate encoding of the data conforming to a desired structure, represented through the prior.
A Statistical Approach to Assessing Neural Network Robustness
Furthermore, it provides an ability to scale to larger networks than formal verification approaches.
On Exploration, Exploitation and Learning in Adaptive Importance Sampling
We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation.
Inference Trees: Adaptive Inference with Exploration
We introduce inference trees (ITs), a new class of inference methods that build on ideas from Monte Carlo tree search to perform adaptive sampling in a manner that balances exploration with exploitation, ensures consistency, and alleviates pathologies in existing adaptive methods.
Hamiltonian Monte Carlo for Probabilistic Programs with Discontinuities
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs).
Nesting Probabilistic Programs
We formalize the notion of nesting probabilistic programming queries and investigate the resulting statistical implications.
Tighter Variational Bounds are Not Necessarily Better
We provide theoretical and empirical evidence that using tighter evidence lower bounds (ELBOs) can be detrimental to the process of learning an inference network by reducing the signal-to-noise ratio of the gradient estimator.
Faithful Inversion of Generative Models for Effective Amortized Inference
Inference amortization methods share information across multiple posterior-inference problems, allowing each to be carried out more efficiently.
On Nesting Monte Carlo Estimators
Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation.
Bayesian Optimization for Probabilistic Programs
We present the first general purpose framework for marginal maximum a posteriori estimation of probabilistic program variables.
Auto-Encoding Sequential Monte Carlo
We build on auto-encoding sequential Monte Carlo (AESMC): a method for model and proposal learning based on maximizing the lower bound to the log marginal likelihood in a broad family of structured probabilistic models.
On the Pitfalls of Nested Monte Carlo
In this paper, we analyse the behaviour of nested Monte Carlo (NMC) schemes, for which classical convergence proofs are insufficient.
Probabilistic structure discovery in time series data
Existing methods for structure discovery in time series data construct interpretable, compositional kernels for Gaussian process regression models.
Interacting Particle Markov Chain Monte Carlo
We introduce interacting particle Markov chain Monte Carlo (iPMCMC), a PMCMC method based on an interacting pool of standard and conditional sequential Monte Carlo samplers.
Canonical Correlation Forests
We introduce canonical correlation forests (CCFs), a new decision tree ensemble method for classification and regression.
