no code implementations • 26 Feb 2025 • Cornelius Emde, Alasdair Paren, Preetham Arvind, Maxime Kayser, Tom Rainforth, Thomas Lukasiewicz, Bernard Ghanem, Philip H. S. Torr, Adel Bibi
Large language models (LLMs) are often deployed to perform constrained tasks, with narrow domains.
no code implementations • 30 Dec 2024 • Freddie Bickford Smith, Jannik Kossen, Eleanor Trollope, Mark van der Wilk, Adam Foster, Tom Rainforth
The ideas of aleatoric and epistemic uncertainty are widely used to reason about the probabilistic predictions of machine-learning models.
1 code implementation • 26 Apr 2024 • Freddie Bickford Smith, Adam Foster, Tom Rainforth
Fully supervised models are predominant in Bayesian active learning.
2 code implementations • 7 Feb 2024 • Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, Tommi Jaakkola
Our approach achieves state-of-the-art co-design performance while allowing the same multimodal model to be used for flexible generation of the sequence or structure.
1 code implementation • 1 Nov 2023 • Tim Reichelt, Luke Ong, Tom Rainforth
We introduce Support Decomposition Variational Inference (SDVI), a new variational inference (VI) approach for probabilistic programs with stochastic support.
1 code implementation • 23 Oct 2023 • Tim Reichelt, Luke Ong, Tom Rainforth
The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path.
1 code implementation • 1 Aug 2023 • Ning Miao, Yee Whye Teh, Tom Rainforth
The recent progress in large language models (LLMs), especially the invention of chain-of-thought prompting, has made it possible to automatically answer questions by stepwise reasoning.
1 code implementation • 23 Jul 2023 • Jannik Kossen, Yarin Gal, Tom Rainforth
The predictions of Large Language Models (LLMs) on downstream tasks often improve significantly when including examples of the input--label relationship in the context.
1 code implementation • 12 Jun 2023 • Guneet S. Dhillon, George Deligiannidis, Tom Rainforth
While conformal predictors reap the benefits of rigorous statistical guarantees on their error frequency, the size of their corresponding prediction sets is critical to their practical utility.
1 code implementation • NeurIPS 2023 • Andrew Campbell, William Harvey, Christian Weilbach, Valentin De Bortoli, Tom Rainforth, Arnaud Doucet
We propose a new class of generative models that naturally handle data of varying dimensionality by jointly modeling the state and dimension of each datapoint.
1 code implementation • 17 Apr 2023 • Freddie Bickford Smith, Andreas Kirsch, Sebastian Farquhar, Yarin Gal, Adam Foster, Tom Rainforth
Information-theoretic approaches to active learning have traditionally focused on maximising the information gathered about the model parameters, most commonly by optimising the BALD score.
no code implementations • 4 Apr 2023 • Mrinank Sharma, Tom Rainforth, Yee Whye Teh, Vincent Fortuin
Conventional Bayesian Neural Networks (BNNs) are unable to leverage unlabelled data to improve their predictions.
no code implementations • 28 Feb 2023 • Tom Rainforth, Adam Foster, Desi R Ivanova, Freddie Bickford Smith
Bayesian experimental design (BED) provides a powerful and general framework for optimizing the design of experiments.
1 code implementation • 27 Feb 2023 • Desi R. Ivanova, Joel Jennings, Tom Rainforth, Cheng Zhang, Adam Foster
We formalize the problem of contextual optimization through the lens of Bayesian experimental design and propose CO-BED -- a general, model-agnostic framework for designing contextual experiments using information-theoretic principles.
2 code implementations • 11 Nov 2022 • Mrinank Sharma, Sebastian Farquhar, Eric Nalisnick, Tom Rainforth
We investigate the benefit of treating all the parameters in a Bayesian neural network stochastically and find compelling theoretical and empirical evidence that this standard construction may be unnecessary.
1 code implementation • 31 May 2022 • Ning Miao, Tom Rainforth, Emile Mathieu, Yann Dubois, Yee Whye Teh, Adam Foster, Hyunjik Kim
We introduce InstaAug, a method for automatically learning input-specific augmentations from data.
2 code implementations • 30 May 2022 • Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, Arnaud Doucet
We provide the first complete continuous time framework for denoising diffusion models of discrete data.
1 code implementation • 14 Feb 2022 • Jannik Kossen, Sebastian Farquhar, Yarin Gal, Tom Rainforth
We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation.
1 code implementation • NeurIPS 2021 • Desi R. Ivanova, Adam Foster, Steven Kleinegesse, Michael U. Gutmann, Tom Rainforth
We introduce implicit Deep Adaptive Design (iDAD), a new method for performing adaptive experiments in real-time with implicit models.
1 code implementation • NeurIPS 2021 • Andrew Campbell, Yuyang Shi, Tom Rainforth, Arnaud Doucet
We present a variational method for online state estimation and parameter learning in state-space models (SSMs), a ubiquitous class of latent variable models for sequential data.
1 code implementation • ICLR 2022 • Ning Miao, Emile Mathieu, N. Siddharth, Yee Whye Teh, Tom Rainforth
InteL-VAEs use an intermediary set of latent variables to control the stochasticity of the encoding process, before mapping these in turn to the latent representation using a parametric function that encapsulates our desired inductive bias(es).
1 code implementation • ICLR 2022 • Tom Joy, Yuge Shi, Philip H. S. Torr, Tom Rainforth, Sebastian M. Schmon, N. Siddharth
Here we introduce a novel alternative, the MEME, that avoids such explicit combinations by repurposing semi-supervised VAEs to combine information between modalities implicitly through mutual supervision.
no code implementations • 22 Jun 2021 • Andreas Kirsch, Tom Rainforth, Yarin Gal
Expanding on MacKay (1992), we argue that conventional model-based methods for active learning - like BALD - have a fundamental shortfall: they fail to directly account for the test-time distribution of the input variables.
1 code implementation • NeurIPS 2021 • Jin Xu, Hyunjik Kim, Tom Rainforth, Yee Whye Teh
We use these layers to construct group equivariant autoencoders (GAEs) that allow us to learn low-dimensional equivariant representations.
no code implementations • pproximateinference AABI Symposium 2021 • Tim Reichelt, Adam Goliński, Luke Ong, Tom Rainforth
We show that the standard computational pipeline of probabilistic programming systems (PPSs) can be inefficient for estimating expectations and introduce the concept of expectation programming to address this.
3 code implementations • NeurIPS 2021 • Jannik Kossen, Neil Band, Clare Lyle, Aidan N. Gomez, Tom Rainforth, Yarin Gal
We challenge a common assumption underlying most supervised deep learning: that a model makes a prediction depending only on its parameters and the features of a single input.
1 code implementation • 9 Mar 2021 • Jannik Kossen, Sebastian Farquhar, Yarin Gal, Tom Rainforth
While approaches like active learning reduce the number of labels needed for model training, existing literature largely ignores the cost of labeling test data, typically unrealistically assuming large test sets for model evaluation.
1 code implementation • 3 Mar 2021 • Adam Foster, Desi R. Ivanova, Ilyas Malik, Tom Rainforth
We introduce Deep Adaptive Design (DAD), a method for amortizing the cost of adaptive Bayesian experimental design that allows experiments to be run in real-time.
no code implementations • 15 Feb 2021 • Ben Barrett, Alexander Camuto, Matthew Willetts, Tom Rainforth
We introduce an approach for training Variational Autoencoders (VAEs) that are certifiably robust to adversarial attack.
no code implementations • ICLR 2021 • Sebastian Farquhar, Yarin Gal, Tom Rainforth
Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution.
1 code implementation • 1 Nov 2020 • Tim G. J. Rudner, Oscar Key, Yarin Gal, Tom Rainforth
We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) issues.
2 code implementations • ICLR 2021 • Adam Foster, Rattana Pukdee, Tom Rainforth
We propose methods to strengthen the invariance properties of representations obtained by contrastive learning.
1 code implementation • 1 Oct 2020 • David Tolpin, Yuan Zhou, Tom Rainforth, Hongseok Yang
We tackle the problem of conditioning probabilistic programs on distributions of observable variables.
no code implementations • 14 Jul 2020 • Alexander Camuto, Matthew Willetts, Stephen Roberts, Chris Holmes, Tom Rainforth
We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations.
2 code implementations • ICLR 2021 • Tom Joy, Sebastian M. Schmon, Philip H. S. Torr, N. Siddharth, Tom Rainforth
We present a principled approach to incorporating labels in VAEs that captures the rich characteristic information associated with those labels.
1 code implementation • 10 Dec 2019 • Benjie Wang, Stefan Webb, Tom Rainforth
The SRR provides a distinct and complementary measure of robust performance, compared to natural and adversarial risk.
1 code implementation • 1 Nov 2019 • Adam Foster, Martin Jankowiak, Matthew O'Meara, Yee Whye Teh, Tom Rainforth
We introduce a fully stochastic gradient based approach to Bayesian optimal experimental design (BOED).
no code implementations • ICML 2020 • Yuan Zhou, Hongseok Yang, Yee Whye Teh, Tom Rainforth
Universal probabilistic programming systems (PPSs) provide a powerful framework for specifying rich probabilistic models.
1 code implementation • 20 Oct 2019 • Saeid Naderiparizi, Adam Ścibior, Andreas Munk, Mehrdad Ghadiri, Atılım Güneş Baydin, Bradley Gram-Hansen, Christian Schroeder de Witt, Robert Zinkov, Philip H. S. Torr, Tom Rainforth, Yee Whye Teh, Frank Wood
Naive approaches to amortized inference in probabilistic programs with unbounded loops can produce estimators with infinite variance.
no code implementations • pproximateinference AABI Symposium 2019 • Bradley Gram-Hansen, Christian Schroeder de Witt, Robert Zinkov, Saeid Naderiparizi, Adam Scibior, Andreas Munk, Frank Wood, Mehrdad Ghadiri, Philip Torr, Yee Whye Teh, Atilim Gunes Baydin, Tom Rainforth
We introduce two approaches for conducting efficient Bayesian inference in stochastic simulators containing nested stochastic sub-procedures, i. e., internal procedures for which the density cannot be calculated directly such as rejection sampling loops.
1 code implementation • 18 Jul 2019 • Adam Goliński, Frank Wood, Tom Rainforth
At runtime, samples are produced separately from each amortized proposal, before being combined to an overall estimate of the expectation.
no code implementations • ICLR 2021 • Matthew Willetts, Alexander Camuto, Tom Rainforth, Stephen Roberts, Chris Holmes
We make significant advances in addressing this issue by introducing methods for producing adversarially robust VAEs.
no code implementations • NeurIPS 2019 • Francesco Locatello, Gabriele Abbati, Tom Rainforth, Stefan Bauer, Bernhard Schölkopf, Olivier Bachem
Recently there has been a significant interest in learning disentangled representations, as they promise increased interpretability, generalization to unseen scenarios and faster learning on downstream tasks.
no code implementations • 29 May 2019 • Bradley Gram-Hansen, Christian Schröder de Witt, Tom Rainforth, Philip H. S. Torr, Yee Whye Teh, Atılım Güneş Baydin
Epidemiology simulations have become a fundamental tool in the fight against the epidemics of various infectious diseases like AIDS and malaria.
1 code implementation • NeurIPS 2019 • Adam Foster, Martin Jankowiak, Eli Bingham, Paul Horsfall, Yee Whye Teh, Tom Rainforth, Noah Goodman
Bayesian optimal experimental design (BOED) is a principled framework for making efficient use of limited experimental resources.
1 code implementation • 6 Mar 2019 • Yuan Zhou, Bradley J. Gram-Hansen, Tobias Kohn, Tom Rainforth, Hongseok Yang, Frank Wood
We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables.
1 code implementation • 6 Dec 2018 • Emile Mathieu, Tom Rainforth, N. Siddharth, Yee Whye Teh
We develop a generalisation of disentanglement in VAEs---decomposition of the latent representation---characterising it as the fulfilment of two factors: a) the latent encodings of the data having an appropriate level of overlap, and b) the aggregate encoding of the data conforming to a desired structure, represented through the prior.
1 code implementation • ICLR 2019 • Stefan Webb, Tom Rainforth, Yee Whye Teh, M. Pawan Kumar
Furthermore, it provides an ability to scale to larger networks than formal verification approaches.
no code implementations • 31 Oct 2018 • Xiaoyu Lu, Tom Rainforth, Yuan Zhou, Jan-Willem van de Meent, Yee Whye Teh
We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation.
no code implementations • 25 Jun 2018 • Tom Rainforth, Yuan Zhou, Xiaoyu Lu, Yee Whye Teh, Frank Wood, Hongseok Yang, Jan-Willem van de Meent
We introduce inference trees (ITs), a new class of inference methods that build on ideas from Monte Carlo tree search to perform adaptive sampling in a manner that balances exploration with exploitation, ensures consistency, and alleviates pathologies in existing adaptive methods.
1 code implementation • 7 Apr 2018 • Bradley Gram-Hansen, Yuan Zhou, Tobias Kohn, Tom Rainforth, Hongseok Yang, Frank Wood
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs).
no code implementations • 16 Mar 2018 • Tom Rainforth
We formalize the notion of nesting probabilistic programming queries and investigate the resulting statistical implications.
3 code implementations • ICML 2018 • Tom Rainforth, Adam R. Kosiorek, Tuan Anh Le, Chris J. Maddison, Maximilian Igl, Frank Wood, Yee Whye Teh
We provide theoretical and empirical evidence that using tighter evidence lower bounds (ELBOs) can be detrimental to the process of learning an inference network by reducing the signal-to-noise ratio of the gradient estimator.
no code implementations • NeurIPS 2018 • Stefan Webb, Adam Golinski, Robert Zinkov, N. Siddharth, Tom Rainforth, Yee Whye Teh, Frank Wood
Inference amortization methods share information across multiple posterior-inference problems, allowing each to be carried out more efficiently.
no code implementations • ICML 2018 • Tom Rainforth, Robert Cornish, Hongseok Yang, Andrew Warrington, Frank Wood
Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation.
2 code implementations • NeurIPS 2016 • Tom Rainforth, Tuan Anh Le, Jan-Willem van de Meent, Michael A. Osborne, Frank Wood
We present the first general purpose framework for marginal maximum a posteriori estimation of probabilistic program variables.
1 code implementation • ICLR 2018 • Tuan Anh Le, Maximilian Igl, Tom Rainforth, Tom Jin, Frank Wood
We build on auto-encoding sequential Monte Carlo (AESMC): a method for model and proposal learning based on maximizing the lower bound to the log marginal likelihood in a broad family of structured probabilistic models.
no code implementations • 3 Dec 2016 • Tom Rainforth, Robert Cornish, Hongseok Yang, Frank Wood
In this paper, we analyse the behaviour of nested Monte Carlo (NMC) schemes, for which classical convergence proofs are insufficient.
no code implementations • 21 Nov 2016 • David Janz, Brooks Paige, Tom Rainforth, Jan-Willem van de Meent, Frank Wood
Existing methods for structure discovery in time series data construct interpretable, compositional kernels for Gaussian process regression models.
1 code implementation • 16 Feb 2016 • Tom Rainforth, Christian A. Naesseth, Fredrik Lindsten, Brooks Paige, Jan-Willem van de Meent, Arnaud Doucet, Frank Wood
We introduce interacting particle Markov chain Monte Carlo (iPMCMC), a PMCMC method based on an interacting pool of standard and conditional sequential Monte Carlo samplers.
3 code implementations • 20 Jul 2015 • Tom Rainforth, Frank Wood
We introduce canonical correlation forests (CCFs), a new decision tree ensemble method for classification and regression.