Search Results for author:
Found 110 papers, 57 papers with code
JoLT: Joint Probabilistic Predictions on Tabular Data Using LLMs
We introduce a simple method for probabilistic predictions on tabular data based on Large Language Models (LLMs) called JoLT (Joint LLM Process for Tabular data).
Spectral-factorized Positive-definite Curvature Learning for NN Training
Many training methods, such as Adam(W) and Shampoo, learn a positive-definite curvature matrix and apply an inverse root before preconditioning.
Tighter sparse variational Gaussian processes
Extensive experiments on regression benchmarks, classification, and latent variable models demonstrate that the proposed approximation consistently matches or outperforms standard sparse variational GPs while maintaining the same computational cost.
Efficient Few-Shot Continual Learning in Vision-Language Models
Experimental results on VQA tasks in the few-shot continual learning setting, validate LoRSU's scalability, efficiency, and effectiveness, making it a compelling solution for image encoder adaptation in resource-constrained environments.
Refined climatologies of future precipitation over High Mountain Asia using probabilistic ensemble learning
While multi-model ensembles are known to improve the predictive accuracy and analysis of future climate projections, consensus regarding how models are aggregated is lacking.
On conditional diffusion models for PDE simulations
Modelling partial differential equations (PDEs) is of crucial importance in science and engineering, and it includes tasks ranging from forecasting to inverse problems, such as data assimilation.
Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Data
Recently, transformer-based approaches have shown great promise in a range of weather forecasting problems.
Linear Transformer Topological Masking with Graph Random Features
When training transformers on graph-structured data, incorporating information about the underlying topology is crucial for good performance.
AI for operational methane emitter monitoring from space
Mitigating methane emissions is the fastest way to stop global warming in the short-term and buy humanity time to decarbonise.
In-Context In-Context Learning with Transformer Neural Processes
Standard NP architectures, such as the convolutional conditional NP (ConvCNP) or the family of transformer neural processes (TNPs), are not capable of in-context in-context learning, as they are only able to condition on a single dataset.
Bayesian Circular Regression with von Mises Quasi-Processes
The need for regression models to predict circular values arises in many scientific fields.
Approximately Equivariant Neural Processes
Our approach is agnostic to both the choice of symmetry group and model architecture, making it widely applicable.
Translation Equivariant Transformer Neural Processes
Notably, the posterior prediction maps for data that are stationary -- a common assumption in spatio-temporal modelling -- exhibit translation equivariance.
Noise-Aware Differentially Private Regression via Meta-Learning
One approach to mitigating this issue is pre-training models on simulated data before DP learning on the private data.
Fearless Stochasticity in Expectation Propagation
We use this insight to motivate two new EP variants, with updates that are particularly well-suited to MC estimation.
Variance-Reducing Couplings for Random Features
We reach surprising conclusions about the benefits and limitations of variance reduction as a paradigm, showing that other properties of the coupling should be optimised for attention estimation in efficient transformers.
LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language
Machine learning practitioners often face significant challenges in formally integrating their prior knowledge and beliefs into predictive models, limiting the potential for nuanced and context-aware analyses.
A Foundation Model for the Earth System
Reliable forecasts of the Earth system are crucial for human progress and safety from natural disasters.
Aardvark weather: end-to-end data-driven weather forecasting
Local station forecasts are skillful up to ten days lead time and achieve comparable and often lower errors than a post-processed global NWP baseline and a state-of-the-art end-to-end forecasting system with input from human forecasters.
A Generative Model of Symmetry Transformations
Correctly capturing the symmetry transformations of data can lead to efficient models with strong generalization capabilities, though methods incorporating symmetries often require prior knowledge.
SportsNGEN: Sustained Generation of Realistic Multi-player Sports Gameplay
By training on a large database of professional tennis tracking data, we demonstrate that simulations produced by SportsNGEN can be used to predict the outcomes of rallies, determine the best shot choices at any point, and evaluate counterfactual or what if scenarios to inform coaching decisions and elevate broadcast coverage.
Denoising Diffusion Probabilistic Models in Six Simple Steps
Denoising Diffusion Probabilistic Models (DDPMs) are a very popular class of deep generative model that have been successfully applied to a diverse range of problems including image and video generation, protein and material synthesis, weather forecasting, and neural surrogates of partial differential equations.
Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
Adaptive gradient optimizers like Adam(W) are the default training algorithms for many deep learning architectures, such as transformers.
Transformer Neural Autoregressive Flows
In this paper, we propose a novel solution to these challenges by exploiting transformers to define a new class of neural flows called Transformer Neural Autoregressive Flows (T-NAFs).
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
Second-order methods such as KFAC can be useful for neural net training.
Identifiable Feature Learning for Spatial Data with Nonlinear ICA
In this paper, we introduce a new nonlinear ICA framework that employs $t$-process (TP) latent components which apply naturally to data with higher-dimensional dependency structures, such as spatial and spatio-temporal data.
Diffusion-Augmented Neural Processes
Over the last few years, Neural Processes have become a useful modelling tool in many application areas, such as healthcare and climate sciences, in which data are scarce and prediction uncertainty estimates are indispensable.
Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
In this work, we identify two different settings of linear weight-sharing layers which motivate two flavours of K-FAC -- $\textit{expand}$ and $\textit{reduce}$.
Sim2Real for Environmental Neural Processes
On held-out weather stations, Sim2Real training substantially outperforms the same model architecture trained only with reanalysis data or only with station data, showing that reanalysis data can serve as a stepping stone for learning from real observations.
Optimising Distributions with Natural Gradient Surrogates
In this work we propose a novel technique for tackling such issues, which involves reframing the optimisation as one with respect to the parameters of a surrogate distribution, for which computing the natural gradient is easy.
Geometric Neural Diffusion Processes
In this work, we extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling.
Beyond Intuition, a Framework for Applying GPs to Real-World Data
Gaussian Processes (GPs) offer an attractive method for regression over small, structured and correlated datasets.
Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation
Despite its simplicity this baseline is competitive with meta-learning methods on a variety of conditions and is able to imitate target policies trained on unseen variations of the original environment.
Few-Shot Image Classification
Few-Shot Imitation Learning
+4
An Introduction to Transformers
The transformer is a neural network component that can be used to learn useful representations of sequences or sets of data-points.
Autoregressive Conditional Neural Processes
Our work provides an example of how ideas from neural distribution estimation can benefit neural processes, and motivates research into the AR deployment of other neural process models.
First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning
In this work, we develop a baseline method, First Session Adaptation (FSA), that sheds light on the efficacy of existing CIL approaches and allows us to assess the relative performance contributions from head and body adaption.
Adversarial Attacks are a Surprisingly Strong Baseline for Poisoning Few-Shot Meta-Learners
This paper examines the robustness of deployed few-shot meta-learning systems when they are fed an imperceptibly perturbed few-shot dataset.
Environmental Sensor Placement with Convolutional Gaussian Neural Processes
This paper proposes using a convolutional Gaussian neural process (ConvGNP) to address these issues.
Ice Core Dating using Probabilistic Programming
Ice cores record crucial information about past climate.
Differentially private partitioned variational inference
In this paper, we present differentially private partitioned variational inference, the first general framework for learning a variational approximation to a Bayesian posterior distribution in the federated learning setting while minimising the number of communication rounds and providing differential privacy guarantees for data subjects.
Kernel Learning for Explainable Climate Science
We account for the spatial variation in precipitation with a non-stationary Gibbs kernel parameterised with an input dependent lengthscale.
The Neural Process Family: Survey, Applications and Perspectives
We shed light on their potential to bring several recent advances in other deep learning domains under one umbrella.
Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification
In this paper we push this Pareto frontier in the few-shot image classification setting with a key contribution: a new adaptive block called Contextual Squeeze-and-Excitation (CaSE) that adjusts a pretrained neural network on a new task to significantly improve performance with a single forward pass of the user data (context).
Ranked #5 on
Few-Shot Image Classification
on Meta-Dataset
Multi-disciplinary fairness considerations in machine learning for clinical trials
While interest in the application of machine learning to improve healthcare has grown tremendously in recent years, a number of barriers prevent deployment in medical practice.
Practical Conditional Neural Processes Via Tractable Dependent Predictions
Existing approaches which model output dependencies, such as Neural Processes (NPs; Garnelo et al., 2018b) or the FullConvGNP (Bruinsma et al., 2021), are either complicated to train or prohibitively expensive.
Modelling Non-Smooth Signals with Complex Spectral Structure
The Gaussian Process Convolution Model (GPCM; Tobar et al., 2015a) is a model for signals with complex spectral structure.
Partitioned Variational Inference: A Framework for Probabilistic Federated Learning
Variational inference (VI) has become the method of choice for fitting many modern probabilistic models.
Memory Efficient Meta-Learning with Large Images
This limitation arises because a task's entire support set, which can contain up to 1000 images, must be processed before an optimization step can be taken.
Continual Novelty Detection
This work identifies the crucial link between the two problems and investigates the Novelty Detection problem under the Continual Learning setting.
Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processes
Pseudo-point approximations, one of the gold-standard methods for scaling GPs to large data sets, are well suited for handling off-the-grid spatial data.
How Tight Can PAC-Bayes be in the Small Data Regime?
Interestingly, this lower bound recovers the Chernoff test set bound if the posterior is equal to the prior.
Contextual HyperNetworks for Novel Feature Adaptation
While deep learning has obtained state-of-the-art results in many applications, the adaptation of neural network architectures to incorporate new output features remains a challenge, as neural networks are commonly trained to produce a fixed output dimension.
Bayesian Neural Network Priors Revisited
Isotropic Gaussian priors are the de facto standard for modern Bayesian neural network inference.
Convolutional conditional neural processes for local climate downscaling
A new model is presented for multisite statistical downscaling of temperature and precipitation using convolutional conditional neural processes (convCNPs).
The Gaussian Neural Process
Neural Processes (NPs; Garnelo et al., 2018a, b) are a rich class of models for meta-learning that map data sets directly to predictive stochastic processes.
Generalized Variational Continual Learning
One strand of research has used probabilistic regularization for continual learning, with two of the main approaches in this vein being Online Elastic Weight Consolidation (Online EWC) and Variational Continual Learning (VCL).
Sparse Gaussian Process Variational Autoencoders
Large, multi-dimensional spatio-temporal datasets are omnipresent in modern science and engineering.
Interpreting Spatially Infinite Generative Models
In this paper we provide a firm theoretical interpretation for infinite spatial generation, by drawing connections to spatial stochastic processes.
Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge
In this competition, participants will focus on the students' answer records to these multiple-choice diagnostic questions, with the aim of 1) accurately predicting which answers the students provide; 2) accurately predicting which questions have high quality; and 3) determining a personalized sequence of questions for each student that best predicts the student's answers.
Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes
Stationary stochastic processes (SPs) are a key component of many probabilistic models, such as those for off-the-grid spatio-temporal data.
Continual Deep Learning by Functional Regularisation of Memorable Past
Continually learning new skills is important for intelligent systems, yet standard deep learning methods suffer from catastrophic forgetting of the past.
TaskNorm: Rethinking Batch Normalization for Meta-Learning
Modern meta-learning approaches for image classification rely on increasingly deep networks to achieve state-of-the-art performance, making batch normalization an essential component of meta-learning pipelines.
Icebreaker: Element-wise Efficient Information Acquisition with a Bayesian Deep Latent Gaussian Model
In this paper, we address the ice-start problem, i. e., the challenge of deploying machine learning models when only a little or no training data is initially available, and acquiring each feature element of data is associated with costs.
Semi-supervised Bootstrapping of Dialogue State Trackers for Task Oriented Modelling
Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels.
Differentially Private Federated Variational Inference
This setting is known as federated learning, in which privacy is a key concern.
Continual Learning with Adaptive Weights (CLAW)
Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner.
Scalable Exact Inference in Multi-Output Gaussian Processes
Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling.
Convolutional Conditional Neural Processes
We introduce the Convolutional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data.
Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations
Second, we demonstrate that the proposed prior encourages a disentangled latent representation which facilitates learning of disentangled representations.
On the Expressiveness of Approximate Inference in Bayesian Neural Networks
While Bayesian neural networks (BNNs) hold the promise of being flexible, well-calibrated statistical models, inference often requires approximations whose consequences are poorly understood.
'In-Between' Uncertainty in Bayesian Neural Networks
We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks.
Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes
We introduce a conditional neural process based approach to the multi-task classification setting for this purpose, and establish connections to the meta-learning and few-shot learning literature.
Ranked #6 on
Few-Shot Image Classification
on Meta-Dataset Rank
Practical Deep Learning with Bayesian Principles
Importantly, the benefits of Bayesian principles are preserved: predictive probabilities are well-calibrated, uncertainties on out-of-distribution data are improved, and continual-learning performance is boosted.
Fast computation of loudness using a deep neural network
The DNN was trained using the output of a more complex model, called the Cambridge loudness model.
Improving and Understanding Variational Continual Learning
In the continual learning setting, tasks are encountered sequentially.
Geometrically Coupled Monte Carlo Sampling
Monte Carlo sampling in high-dimensional, low-sample settings is important in many machine learning tasks.
Partitioned Variational Inference: A unified framework encompassing federated and continual learning
Second, the granularity of the updates e. g. whether the updates are local to each data point and employ message passing or global.
Infinite-Horizon Gaussian Processes
The complexity is still cubic in the state dimension $m$ which is an impediment to practical application.
Deterministic Variational Inference for Robust Bayesian Neural Networks
We provide two innovations that aim to turn VB into a robust inference tool for Bayesian neural networks: first, we introduce a novel deterministic method to approximate moments in neural networks, eliminating gradient variance; second, we introduce a hierarchical prior for parameters and a novel Empirical Bayes procedure for automatically selecting prior variances.
Meta-Learning Probabilistic Inference For Prediction
2) We introduce VERSA, an instance of the framework employing a flexible and versatile amortization network that takes few-shot learning datasets as inputs, with arbitrary numbers of shots, and outputs a distribution over task-specific parameters in a single forward pass.
Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning
Here, we propose a general framework for nonlinear ICA, which, as a special case, can make use of temporal structure.
Gaussian Process Behaviour in Wide Deep Neural Networks
Whilst deep neural networks have shown great empirical success, there is still much work to be done to understand their theoretical properties.
Structured Evolution with Compact Architectures for Scalable Policy Optimization
We present a new method of blackbox optimization via gradient approximation with the use of structured random orthogonal matrices, providing more accurate estimators than baselines and with provable theoretical guarantees.
The Mirage of Action-Dependent Baselines in Reinforcement Learning
Policy gradient methods are a widely used class of model-free reinforcement learning algorithms where a state-dependent baseline is used to reduce gradient estimator variance.
Learning Causally-Generated Stationary Time Series
We present the Causal Gaussian Process Convolution Model (CGPCM), a doubly nonparametric model for causal, spectrally complex dynamical phenomena.
The Gaussian Process Autoregressive Regression Model (GPAR)
Multi-output regression models must exploit dependencies between outputs to maximise predictive performance.
Conditional Density Estimation with Bayesian Normalising Flows
Modeling complex conditional distributions is critical in a variety of settings.
Variational Continual Learning
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks.
Discriminative k-shot learning using probabilistic models
The goal is to generalise from an initial large-scale classification task to a separate task comprising new classes and small numbers of examples.
Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning
Off-policy model-free deep reinforcement learning methods using previously collected data can improve sample efficiency over on-policy policy gradient techniques.
Gradient Estimators for Implicit Models
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research.
Streaming Sparse Gaussian Process Approximations
Sparse pseudo-point approximations for Gaussian process (GP) models provide a suite of methods that support deployment of GPs in the large data regime and enable analytic intractabilities to be sidestepped.
Approximate Inference with Amortised MCMC
We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler.
Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
This paper proposes a general method for improving the structure and quality of sequences generated by a recurrent neural network (RNN), while maintaining information originally learned from data, as well as sample diversity.
Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
We analyze the connection between Q-Prop and existing model-free algorithms, and use control variate theory to derive two variants of Q-Prop with conservative and aggressive adaptation.
A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation
Unlike much of the previous venerable work in this area, the new framework is built on standard methods for approximate inference (variational free-energy, EP and Power EP methods) rather than employing approximations to the probabilistic generative model itself.
The Multivariate Generalised von Mises distribution: Inference and applications
First we introduce a new multivariate distribution over circular variables, called the multivariate Generalised von Mises (mGvM) distribution.
Deep Gaussian Processes for Regression using Approximate Expectation Propagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers.
Rényi Divergence Variational Inference
This paper introduces the variational R\'enyi bound (VR) that extends traditional variational inference to R\'enyi's alpha-divergences.
Learning Stationary Time Series using Gaussian Processes with Nonparametric Kernels
We introduce the Gaussian Process Convolution Model (GPCM), a two-stage nonparametric generative procedure to model stationary signals as the convolution between a continuous-time white-noise process and a continuous-time linear filter drawn from Gaussian process.
Training Deep Gaussian Processes using Stochastic Expectation Propagation and Probabilistic Backpropagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers.
Stochastic Expectation Propagation for Large Scale Gaussian Process Classification
A method for large scale Gaussian process classification has been recently proposed based on expectation propagation (EP).
Black-box $α$-divergence Minimization
Black-box alpha (BB-$\alpha$) is a new approximate inference method based on the minimization of $\alpha$-divergences.
Denoising without access to clean data using a partitioned autoencoder
Training a denoising autoencoder neural network requires access to truly clean data, a requirement which is often impractical.
Stochastic Expectation Propagation
Expectation propagation (EP) is a deterministic approximation algorithm that is often used to perform approximate Bayesian parameter learning.
Neural Adaptive Sequential Monte Carlo
Experiments indicate that NASMC significantly improves inference in a non-linear state space model outperforming adaptive proposal methods including the Extended Kalman and Unscented Particle Filters.
On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes
We then discuss augmented index sets and show that, contrary to previous works, marginal consistency of augmentation is not enough to guarantee consistency of variational inference with the original model.
Tree-structured Gaussian Process Approximations
Gaussian process regression can be accelerated by constructing a small pseudo-dataset to summarise the observed data.
Target Fishing: A Single-Label or Multi-Label Problem?
According to Cobanoglu et al and Murphy, it is now widely acknowledged that the single target paradigm (one protein or target, one disease, one drug) that has been the dominant premise in drug development in the recent past is untenable.
