Search Results for author:
Found 98 papers, 51 papers with code
Compression with Bayesian Implicit Neural Representations
Many common types of data can be represented as functions that map coordinates to signal values, such as pixel locations to RGB values in the case of an image.
Fast and Painless Image Reconstruction in Deep Image Prior Subspaces
The deep image prior (DIP) is a state-of-the-art unsupervised approach for solving linear inverse problems in imaging.
normflows: A PyTorch Package for Normalizing Flows
It allows to build normalizing flow models from a suite of base distributions, flow layers, and neural networks.
Sampling-based inference for large linear models, with application to linearised Laplace
Large-scale linear models are ubiquitous throughout machine learning, with contemporary application as surrogate models for neural network uncertainty quantification; that is, the linearised Laplace method.
Flow Annealed Importance Sampling Bootstrap
Normalizing flows are tractable density models that can approximate complicated target distributions, e. g. Boltzmann distributions of physical systems.
Bayesian Experimental Design for Computed Tomography with the Linearised Deep Image Prior
We investigate adaptive design based on a single sparse pilot scan for generating effective scanning strategies for computed tomography reconstruction.
Adapting the Linearised Laplace Model Evidence for Modern Deep Learning
The linearised Laplace method for estimating model uncertainty has received renewed attention in the Bayesian deep learning community.
Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction
We propose Adaptive Deep Kernel Fitting with Implicit Function Theorem (ADKF-IFT), a novel framework for learning deep kernel Gaussian processes (GPs) by interpolating between meta-learning and conventional deep kernel learning.
Uncertainty Estimation for Computed Tomography with a Linearised Deep Image Prior
Existing deep-learning based tomographic image reconstruction methods do not provide accurate estimates of reconstruction uncertainty, hindering their real-world deployment.
Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo
Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features.
Addressing Bias in Active Learning with Depth Uncertainty Networks... or Not
Farquhar et al. [2021] show that correcting for active learning bias with underparameterised models leads to improved downstream performance.
Depth Uncertainty Networks for Active Learning
In active learning, the size and complexity of the training dataset changes over time.
Functional Variational Inference based on Stochastic Process Generators
In this paper, we propose a new solution to this problem called Functional Variational Inference (FVI).
Bootstrap Your Flow
Normalizing flows are flexible, parameterized distributions that can be used to approximate expectations from intractable distributions via importance sampling.
A Probabilistic Deep Image Prior over Image Space
The deep image prior regularises under-specified image reconstruction problems by reparametrising the target image as the output of a CNN.
Linearised Laplace Inference in Networks with Normalisation Layers and the Neural g-Prior
We show that for neural networks (NN) with normalisation layers, i. e. batch norm, layer norm, or group norm, the Laplace model evidence does not approximate the volume of a posterior mode and is thus unsuitable for model selection.
Resampling Base Distributions of Normalizing Flows
Normalizing flows are a popular class of models for approximating probability distributions.
Ranked #42 on
Image Generation
on CIFAR-10
(bits/dimension metric)
DOCKSTRING: easy molecular docking yields better benchmarks for ligand design
The field of machine learning for drug discovery is witnessing an explosion of novel methods.
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation
Machine learning training methods depend plentifully and intricately on hyperparameters, motivating automated strategies for their optimisation.
Action-Sufficient State Representation Learning for Control with Structural Constraints
Perceived signals in real-world scenarios are usually high-dimensional and noisy, and finding and using their representation that contains essential and sufficient information required by downstream decision-making tasks will help improve computational efficiency and generalization ability in the tasks.
Invariant Causal Representation Learning for Out-of-Distribution Generalization
Extensive experiments on both synthetic and real-world datasets show that our approach outperforms a variety of baseline methods.
A Fresh Look at De Novo Molecular Design Benchmarks
De novo molecular design is a thriving research area in machine learning (ML) that lacks ubiquitous, high-quality, standardized benchmark tasks.
Improving black-box optimization in VAE latent space using decoder uncertainty
Optimization in the latent space of variational autoencoders is a promising approach to generate high-dimensional discrete objects that maximize an expensive black-box property (e. g., drug-likeness in molecular generation, function approximation with arithmetic expressions).
Contextual HyperNetworks for Novel Feature Adaptation
While deep learning has obtained state-of-the-art results in many applications, the adaptation of neural network architectures to incorporate new output features remains a challenge, as neural networks are commonly trained to produce a fixed output dimension.
Active Slices for Sliced Stein Discrepancy
First, we provide theoretical results stating that the requirement of using optimal slicing directions in the kernelized version of SSD can be relaxed, validating the resulting discrepancy with finite random slicing directions.
Activation-level uncertainty in deep neural networks
Current approaches for uncertainty estimation in deep learning often produce too confident results.
Gradient-based tuning of Hamiltonian Monte Carlo hyperparameters
Existing approaches for automating this task either optimise a proxy for mixing speed or consider the HMC chain as an implicit variational distribution and optimize a tractable lower bound that is too loose to be useful in practice.
Invariant Causal Representation Learning
As an alternative, we propose Invariant Causal Representation Learning (ICRL), a learning paradigm that enables out-of-distribution generalization in the nonlinear setting (i. e., nonlinear representations and nonlinear classifiers).
Barking up the right tree: an approach to search over molecule synthesis DAGs
When designing new molecules with particular properties, it is not only important what to make but crucially how to make it.
Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation
We propose counterfactual RL algorithms to learn both population-level and individual-level policies.
BSODA: A Bipartite Scalable Framework for Online Disease Diagnosis
To address the challenge, we propose a non-RL Bipartite Scalable framework for Online Disease diAgnosis, called BSODA.
Symmetry-Aware Actor-Critic for 3D Molecular Design
Automating molecular design using deep reinforcement learning (RL) has the potential to greatly accelerate the search for novel materials.
Bayesian Deep Learning via Subnetwork Inference
In particular, we implement subnetwork linearized Laplace as a simple, scalable Bayesian deep learning method: We first obtain a MAP estimate of all weights and then infer a full-covariance Gaussian posterior over a subnetwork using the linearized Laplace approximation.
Compressing Images by Encoding Their Latent Representations with Relative Entropy Coding
Variational Autoencoders (VAEs) have seen widespread use in learned image compression.
Expressive yet Tractable Bayesian Deep Learning via Subnetwork Inference
In particular, we develop a practical and scalable Bayesian deep learning method that first trains a point estimate, and then infers a full covariance Gaussian posterior approximation over a subnetwork.
Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge
In this competition, participants will focus on the students' answer records to these multiple-choice diagnostic questions, with the aim of 1) accurately predicting which answers the students provide; 2) accurately predicting which questions have high quality; and 3) determining a personalized sequence of questions for each student that best predicts the student's answers.
DRIFT: Deep Reinforcement Learning for Functional Software Testing
Efficient software testing is essential for productive software development and reliable user experiences.
Sliced Kernelized Stein Discrepancy
Kernelized Stein discrepancy (KSD), though being extensively used in goodness-of-fit tests and model learning, suffers from the curse-of-dimensionality.
VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data
Deep generative models often perform poorly in real-world applications due to the heterogeneity of natural data sets.
Predictive Complexity Priors
For this reason, we propose predictive complexity priors: a functional prior that is defined by comparing the model's predictions to those of a reference model.
Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining
We introduce an improved method for efficient black-box optimization, which performs the optimization in the low-dimensional, continuous latent manifold learned by a deep generative model.
Ranked #1 on
Molecular Graph Generation
on ZINC
Depth Uncertainty in Neural Networks
Existing methods for estimating uncertainty in deep learning tend to require multiple forward passes, making them unsuitable for applications where computational resources are limited.
Getting a CLUE: A Method for Explaining Uncertainty Estimates
Both uncertainty estimation and interpretability are important factors for trustworthy machine learning systems.
Excursion Search for Constrained Bayesian Optimization under a Limited Budget of Failures
When learning to ride a bike, a child falls down a number of times before achieving the first success.
Reinforcement Learning for Molecular Design Guided by Quantum Mechanics
Automating molecular design using deep reinforcement learning (RL) holds the promise of accelerating the discovery of new chemical compounds.
Variational Depth Search in ResNets
One-shot neural architecture search allows joint learning of weights and network architecture, reducing computational cost.
Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
Despite their successes, deep neural networks may make unreliable predictions when faced with test data drawn from a distribution different to that of the training data, constituting a major problem for AI safety.
Icebreaker: Element-wise Efficient Information Acquisition with a Bayesian Deep Latent Gaussian Model
In this paper, we address the ice-start problem, i. e., the challenge of deploying machine learning models when only a little or no training data is initially available, and acquiring each feature element of data is associated with costs.
Refining the variational posterior through iterative optimization
Variational inference (VI) is a popular approach for approximate Bayesian inference that is particularly promising for highly parameterized models such as deep neural networks.
A Generative Model for Molecular Distance Geometry
Great computational effort is invested in generating equilibrium states for molecular systems using, for example, Markov chain Monte Carlo.
Compression without Quantization
Standard compression algorithms work by mapping an image to discrete code using an encoder from which the original image can be reconstructed through a decoder.
Icebreaker: Element-wise Active Information Acquisition with Bayesian Deep Latent Gaussian Model
In this paper we introduce the ice-start problem, i. e., the challenge of deploying machine learning models when only little or no training data is initially available, and acquiring each feature element of data is associated with costs.
Bayesian Batch Active Learning as Sparse Subset Approximation
Leveraging the wealth of unlabeled data produced in recent years provides great potential for improving supervised models.
'In-Between' Uncertainty in Bayesian Neural Networks
We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks.
A Model to Search for Synthesizable Molecules
Deep generative models are able to suggest new organic molecules by generating strings, trees, and graphs representing their structure.
A COLD Approach to Generating Optimal Samples
Carrying out global optimisation is difficult as optimisers are likely to follow gradients into regions of the latent space that the model has not been exposed to during training; samples generated from these regions are likely to be too dissimilar to the training data to be useful.
Interpretable Outcome Prediction with Sparse Bayesian Neural Networks in Intensive Care
However, flexible tools such as artificial neural networks (ANNs) suffer from a lack of interpretability limiting their acceptability to clinicians.
Generating Molecules via Chemical Reactions
We therefore propose a new molecule generation model, mirroring a more realistic real-world process, where reactants are selected and combined to form more complex molecules.
Deconfounding Reinforcement Learning in Observational Settings
Using this benchmark, we demonstrate that the proposed algorithms are superior to traditional RL methods in confounded environments with observational data.
Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
Posterior sampling for reinforcement learning (PSRL) is an effective method for balancing exploration and exploitation in reinforcement learning.
Dropout as a Structured Shrinkage Prior
We propose a novel framework for understanding multiplicative noise in neural networks, considering continuous distributions as well as Bernoulli noise (i. e. dropout).
Deterministic Variational Inference for Robust Bayesian Neural Networks
We provide two innovations that aim to turn VB into a robust inference tool for Bayesian neural networks: first, we introduce a novel deterministic method to approximate moments in neural networks, eliminating gradient variance; second, we introduce a hierarchical prior for parameters and a novel Empirical Bayes procedure for automatically selecting prior variances.
Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters
While deep neural networks are a highly successful model class, their large memory footprint puts considerable strain on energy consumption, communication bandwidth, and storage requirements.
EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE
Many real-life decision-making situations allow further relevant information to be acquired at a specific cost, for example, in assessing the health status of a patient we may decide to take additional measurements such as diagnostic tests or imaging scans before making a final assessment.
Ergodic Measure Preserving Flows
Training probabilistic models with neural network components is intractable in most cases and requires to use approximations such as Markov chain Monte Carlo (MCMC), which is not scalable and requires significant hyper-parameter tuning, or mean-field variational inference (VI), which is biased.
Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
The current state-of-the-art inference method, Variational Inference (VI), employs a Gaussian approximation to the posterior distribution.
Meta-Learning for Stochastic Gradient MCMC
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling.
Variational Implicit Processes
We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables.
Ergodic Inference: Accelerate Convergence by Optimisation
In this work, we aim to improve upon MCMC and VI by a novel hybrid method based on the idea of reducing simulation bias of finite-length MCMC chains using gradient-based optimisation.
A Generative Model For Electron Paths
Chemical reactions can be described as the stepwise redistribution of electrons in molecules.
Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control
In this work we introduce the application of black-box quantum control as an interesting rein- forcement learning problem to the machine learning community.
Deep Gaussian Processes with Decoupled Inducing Inputs
Deep Gaussian Processes (DGP) are hierarchical generalizations of Gaussian Processes (GP) that have proven to work effectively on a multiple supervised regression tasks.
Sensitivity Analysis for Predictive Uncertainty in Bayesian Neural Networks
We derive a novel sensitivity analysis of input variables for predictive epistemic and aleatoric uncertainty.
Learning a Generative Model for Validity in Complex Discrete Structures
This validator provides insight as to how individual sequence elements influence the validity of the overall sequence, and can be used to constrain sequence based models to generate valid sequences -- and thus faithfully model discrete objects.
Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning
Bayesian neural networks with latent variables are scalable and flexible probabilistic models: They account for uncertainty in the estimation of the network weights and, by making use of latent variables, can capture complex noise patterns in the data.
Constrained Bayesian Optimization for Automatic Chemical Design
Automatic Chemical Design is a framework for generating novel molecules with optimized properties.
Actively Learning what makes a Discrete Sequence Valid
As a step towards solving this problem, we propose to learn a deep recurrent validator model.
Bayesian Semisupervised Learning with Deep Generative Models
However, these techniques a) cannot account for model uncertainty in the estimation of the model's discriminative component and b) lack flexibility to capture complex stochastic patterns in the label generation process.
Uncertainty Decomposition in Bayesian Neural Networks with Latent Variables
Bayesian neural networks (BNNs) with latent variables are probabilistic models which can automatically identify complex stochastic patterns in the data.
Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space
These results show that PDTS is a successful solution for large-scale parallel BO.
Grammar Variational Autoencoder
Crucially, state-of-the-art methods often produce outputs that are not valid.
GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
Generative Adversarial Networks (GAN) have limitations when the goal is to generate sequences of discrete elements.
Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
This paper proposes a general method for improving the structure and quality of sequences generated by a recurrent neural network (RNN), while maintaining information originally learned from data, as well as sample diversity.
Automatic chemical design using a data-driven continuous representation of molecules
We report a method to convert discrete representations of molecules to and from a multidimensional continuous representation.
Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks
We present an algorithm for model-based reinforcement learning that combines Bayesian neural networks (BNNs) with random roll-outs and stochastic optimization for policy learning.
Model-based Reinforcement Learning
reinforcement-learning
+2
Deep Gaussian Processes for Regression using Approximate Expectation Propagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers.
A General Framework for Constrained Bayesian Optimization using Information-based Search
Of particular interest to us is to efficiently solve problems with decoupled constraints, in which subsets of the objective and constraint functions may be evaluated independently.
Predictive Entropy Search for Multi-objective Bayesian Optimization
The results show that PESMO produces better recommendations with a smaller number of evaluations of the objectives, and that a decoupled evaluation can lead to improvements in performance, particularly when the number of objectives is large.
Training Deep Gaussian Processes using Stochastic Expectation Propagation and Probabilistic Backpropagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers.
Black-box $α$-divergence Minimization
Black-box alpha (BB-$\alpha$) is a new approximate inference method based on the minimization of $\alpha$-divergences.
Stochastic Expectation Propagation for Large Scale Gaussian Process Classification
A method for large scale Gaussian process classification has been recently proposed based on expectation propagation (EP).
Scalable Gaussian Process Classification via Expectation Propagation
Variational methods have been recently considered for scaling the training process of Gaussian process classifiers to large datasets.
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
In principle, the Bayesian approach to learning neural networks does not have these problems.
Predictive Entropy Search for Bayesian Optimization with Unknown Constraints
Unknown constraints arise in many types of expensive black-box optimization problems.
Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES).
Learning Feature Selection Dependencies in Multi-task Learning
Because the process of estimating feature selection dependencies may suffer from over-fitting in the model proposed, additional data from a multi-task learning scenario are considered for induction.
Gaussian Process Conditional Copulas with Applications to Financial Time Series
The estimation of dependencies between multiple variables is a central problem in the analysis of financial time series.
Dynamic Covariance Models for Multivariate Financial Time Series
The accurate prediction of time-changing covariances is an important problem in the modeling of multivariate financial data.
