Search Results for author: Yarin Gal

Found 128 papers, 59 papers with code

Simple and Scalable Epistemic Uncertainty Estimation Using a Single Deep Deterministic Neural Network

no code implementations ICML 2020 Joost van Amersfoort, Lewis Smith, Yee Whye Teh, Yarin Gal

We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass.

Inter-domain Deep Gaussian Processes with RKHS Fourier Features

no code implementations ICML 2020 Tim G. J. Rudner, Dino Sejdinovic, Yarin Gal

We propose Inter-domain Deep Gaussian Processes with RKHS Fourier Features, an extension of shallow inter-domain GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs) and demonstrate how to leverage existing approximate inference approaches to perform simple and scalable approximate inference on Inter-domain Deep Gaussian Processes.

Gaussian Processes

Scalable Sensitivity and Uncertainty Analysis for Causal-Effect Estimates of Continuous-Valued Interventions

no code implementations21 Apr 2022 Andrew Jesson, Alyson Douglas, Peter Manshausen, Nicolai Meinshausen, Philip Stier, Yarin Gal, Uri Shalit

Here, we develop a continuous treatment-effect marginal sensitivity model (CMSM) and derive bounds that agree with both the observed data and a researcher-defined level of hidden confounding.

Interventions, Where and How? Experimental Design for Causal Models at Scale

no code implementations3 Mar 2022 Panagiotis Tigas, Yashas Annadani, Andrew Jesson, Bernhard Schölkopf, Yarin Gal, Stefan Bauer

Causal discovery from observational and interventional data is challenging due to limited data and non-identifiability which introduces uncertainties in estimating the underlying structural causal model (SCM).

Causal Discovery Experimental Design

Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients

1 code implementation ICLR 2022 Milad Alizadeh, Shyam A. Tailor, Luisa M Zintgraf, Joost van Amersfoort, Sebastian Farquhar, Nicholas Donald Lane, Yarin Gal

Pruning neural networks at initialization would enable us to find sparse models that retain the accuracy of the original network while consuming fewer computational resources for training and inference.

Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation

1 code implementation14 Feb 2022 Jannik Kossen, Sebastian Farquhar, Yarin Gal, Tom Rainforth

We find that ASEs offer greater label-efficiency than the current state-of-the-art when applied to challenging model evaluation problems for deep neural networks.

Active Learning

A Note on "Assessing Generalization of SGD via Disagreement"

no code implementations3 Feb 2022 Andreas Kirsch, Yarin Gal

Jiang et al. (2021) give empirical evidence that the average test error of deep neural networks can be estimated via the prediction disagreement of two separately trained networks.

DARTS without a Validation Set: Optimizing the Marginal Likelihood

no code implementations24 Dec 2021 Miroslav Fil, Binxin Ru, Clare Lyle, Yarin Gal

The success of neural architecture search (NAS) has historically been limited by excessive compute requirements.

Neural Architecture Search

QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation -- Analysis of Ranking Metrics and Benchmarking Results

1 code implementation19 Dec 2021 Raghav Mehta, Angelos Filos, Ujjwal Baid, Chiharu Sako, Richard McKinley, Michael Rebsamen, Katrin Dätwyler, Raphael Meier, Piotr Radojewski, Gowtham Krishnan Murugesan, Sahil Nalawade, Chandan Ganesh, Ben Wagner, Fang F. Yu, Baowei Fei, Ananth J. Madhuranthakam, Joseph A. Maldjian, Laura Daza, Catalina Gómez, Pablo Arbeláez, Chengliang Dai, Shuo Wang, Hadrien Raynaud, Yuanhan Mo, Elsa Angelini, Yike Guo, Wenjia Bai, Subhashis Banerjee, Linmin Pei, Murat AK, Sarahi Rosas-González, Illyess Zemmoura, Clovis Tauber, Minh H. Vu, Tufve Nyholm, Tommy Löfstedt, Laura Mora Ballestar, Veronica Vilaplana, Hugh McHugh, Gonzalo Maso Talou, Alan Wang, Jay Patel, Ken Chang, Katharina Hoebel, Mishka Gidwani, Nishanth Arun, Sharut Gupta, Mehak Aggarwal, Praveer Singh, Elizabeth R. Gerstner, Jayashree Kalpathy-Cramer, Nicolas Boutry, Alexis Huard, Lasitha Vidyaratne, Md Monibor Rahman, Khan M. Iftekharuddin, Joseph Chazalon, Elodie Puybareau, Guillaume Tochon, Jun Ma, Mariano Cabezas, Xavier Llado, Arnau Oliver, Liliana Valencia, Sergi Valverde, Mehdi Amian, Mohammadreza Soltaninejad, Andriy Myronenko, Ali Hatamizadeh, Xue Feng, Quan Dou, Nicholas Tustison, Craig Meyer, Nisarg A. Shah, Sanjay Talbar, Marc-Andr Weber, Abhishek Mahajan, Andras Jakab, Roland Wiest, Hassan M. Fathallah-Shaykh, Arash Nazeri, Mikhail Milchenko, Daniel Marcus, Aikaterini Kotrotsou, Rivka Colen, John Freymann, Justin Kirby, Christos Davatzikos, Bjoern Menze, Spyridon Bakas, Yarin Gal, Tal Arbel

In this study, we explore and evaluate a metric developed during the BraTS 2019-2020 task on uncertainty quantification (QU-BraTS), and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation.

Brain Tumor Segmentation Translation +1

On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations

no code implementations NeurIPS 2021 Tim G. J. Rudner, Cong Lu, Michael Osborne, Yarin Gal, Yee Teh

KL-regularized reinforcement learning from expert demonstrations has proved highly successful in improving the sample efficiency of deep reinforcement learning algorithms, allowing them to be applied to challenging physical real-world tasks.

reinforcement-learning

DeDUCE: Generating Counterfactual Explanations Efficiently

1 code implementation29 Nov 2021 Benedikt Höltgen, Lisa Schut, Jan M. Brauner, Yarin Gal

This is the aim of algorithms generating counterfactual explanations.

Contrastive Representation Learning with Trainable Augmentation Channel

no code implementations15 Nov 2021 Masanori Koyama, Kentaro Minami, Takeru Miyato, Yarin Gal

In contrastive representation learning, data representation is trained so that it can classify the image instances even when the images are altered by augmentations.

Representation Learning

Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data

1 code implementation NeurIPS 2021 Andrew Jesson, Panagiotis Tigas, Joost van Amersfoort, Andreas Kirsch, Uri Shalit, Yarin Gal

We introduce causal, Bayesian acquisition functions grounded in information theory that bias data acquisition towards regions with overlapping support to maximize sample efficiency for learning personalized treatment effects.

Active Learning

Deep Deterministic Uncertainty for Semantic Segmentation

no code implementations29 Oct 2021 Jishnu Mukhoti, Joost van Amersfoort, Philip H. S. Torr, Yarin Gal

We extend Deep Deterministic Uncertainty (DDU), a method for uncertainty estimation using feature space densities, to semantic segmentation.

Semantic Segmentation

Using Non-Linear Causal Models to Study Aerosol-Cloud Interactions in the Southeast Pacific

no code implementations28 Oct 2021 Andrew Jesson, Peter Manshausen, Alyson Douglas, Duncan Watson-Parris, Yarin Gal, Philip Stier

Aerosol-cloud interactions include a myriad of effects that all begin when aerosol enters a cloud and acts as cloud condensation nuclei (CCN).

GeneDisco: A Benchmark for Experimental Design in Drug Discovery

no code implementations ICLR 2022 Arash Mehrjou, Ashkan Soleymani, Andrew Jesson, Pascal Notin, Yarin Gal, Stefan Bauer, Patrick Schwab

GeneDisco contains a curated set of multiple publicly available experimental data sets as well as open-source implementations of state-of-the-art active learning policies for experimental design and exploration.

Active Learning Drug Discovery +1

Quantifying Uncertainty for Machine Learning Based Diagnostic

no code implementations29 Jul 2021 Owen Convery, Lewis Smith, Yarin Gal, Adi Hanuka

Virtual Diagnostic (VD) is a deep learning tool that can be used to predict a diagnostic output.

Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks

2 code implementations15 Jul 2021 Andrey Malinin, Neil Band, Ganshin, Alexander, German Chesnokov, Yarin Gal, Mark J. F. Gales, Alexey Noskov, Andrey Ploskonosov, Liudmila Prokhorenkova, Ivan Provilkov, Vatsal Raina, Vyas Raina, Roginskiy, Denis, Mariya Shmatova, Panos Tigas, Boris Yangel

However, many tasks of practical interest have different modalities, such as tabular data, audio, text, or sensor data, which offer significant challenges involving regression and discrete or continuous structured prediction.

Image Classification Machine Translation +3

Improving black-box optimization in VAE latent space using decoder uncertainty

1 code implementation NeurIPS 2021 Pascal Notin, José Miguel Hernández-Lobato, Yarin Gal

Optimization in the latent space of variational autoencoders is a promising approach to generate high-dimensional discrete objects that maximize an expensive black-box property (e. g., drug-likeness in molecular generation, function approximation with arithmetic expressions).

Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers

no code implementations22 Jun 2021 Andreas Kirsch, Tom Rainforth, Yarin Gal

Expanding on MacKay (1992), we argue that conventional model-based methods for active learning - like BALD - have a fundamental shortfall: they fail to directly account for the test-time distribution of the input variables.

Active Learning

KL Guided Domain Adaptation

no code implementations ICLR 2022 A. Tuan Nguyen, Toan Tran, Yarin Gal, Philip H. S. Torr, Atılım Güneş Baydin

A common approach in the domain adaptation literature is to learn a representation of the input that has the same (marginal) distribution over the source and the target domain.

Domain Adaptation

Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective

no code implementations4 Jun 2021 Lewis Smith, Joost van Amersfoort, Haiwen Huang, Stephen Roberts, Yarin Gal

ResNets constrained to be bi-Lipschitz, that is, approximately distance preserving, have been a crucial component of recently proposed techniques for deterministic uncertainty quantification in neural models.

Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning

2 code implementations NeurIPS 2021 Jannik Kossen, Neil Band, Clare Lyle, Aidan N. Gomez, Tom Rainforth, Yarin Gal

We challenge a common assumption underlying most supervised deep learning: that a model makes a prediction depending only on its parameters and the features of a single input.

3D Part Segmentation

Outcome-Driven Reinforcement Learning via Variational Inference

no code implementations NeurIPS 2021 Tim G. J. Rudner, Vitchyr H. Pong, Rowan Mcallister, Yarin Gal, Sergey Levine

While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it.

reinforcement-learning Variational Inference

Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties

1 code implementation16 Mar 2021 Lisa Schut, Oscar Key, Rory McGrath, Luca Costabello, Bogdan Sacaleanu, Medb Corcoran, Yarin Gal

Counterfactual explanations (CEs) are a practical tool for demonstrating why machine learning classifiers make particular decisions.

Robustness to Pruning Predicts Generalization in Deep Neural Networks

no code implementations10 Mar 2021 Lorenz Kuhn, Clare Lyle, Aidan N. Gomez, Jonas Rothfuss, Yarin Gal

Existing generalization measures that aim to capture a model's simplicity based on parameter counts or norms fail to explain generalization in overparameterized deep neural networks.

Active Testing: Sample-Efficient Model Evaluation

1 code implementation9 Mar 2021 Jannik Kossen, Sebastian Farquhar, Yarin Gal, Tom Rainforth

While approaches like active learning reduce the number of labels needed for model training, existing literature largely ignores the cost of labeling test data, typically unrealistically assuming large test sets for model evaluation.

Active Learning Gaussian Processes

Resolving Causal Confusion in Reinforcement Learning via Robust Exploration

no code implementations ICLR Workshop SSL-RL 2021 Clare Lyle, Amy Zhang, Minqi Jiang, Joelle Pineau, Yarin Gal

To address this, we present a robust exploration strategy which enables causal hypothesis-testing by interaction with the environment.

reinforcement-learning

Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

1 code implementation8 Mar 2021 Andrew Jesson, Sören Mindermann, Yarin Gal, Uri Shalit

We study the problem of learning conditional average treatment effects (CATE) from high-dimensional, observational data with unobserved confounders.

Deep Deterministic Uncertainty: A Simple Baseline

3 code implementations23 Feb 2021 Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip H. S. Torr, Yarin Gal

Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive.

Active Learning

On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty

1 code implementation22 Feb 2021 Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, Yarin Gal

Inducing point Gaussian process approximations are often considered a gold standard in uncertainty estimation since they retain many of the properties of the exact GP and scale to large datasets.

Gaussian Processes General Classification

Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies

1 code implementation16 Feb 2021 Mike Walmsley, Chris Lintott, Tobias Geron, Sandor Kruk, Coleman Krawczyk, Kyle W. Willett, Steven Bamford, Lee S. Kelvin, Lucy Fortson, Yarin Gal, William Keel, Karen L. Masters, Vihang Mehta, Brooke D. Simmons, Rebecca Smethurst, Lewis Smith, Elisabeth M. Baeten, Christine Macmillan

All classifications are used to train an ensemble of Bayesian convolutional neural networks (a state-of-the-art deep learning method) to predict posteriors for the detailed morphology of all 314, 000 galaxies.

Domain Invariant Representation Learning with Domain Density Transformations

1 code implementation NeurIPS 2021 A. Tuan Nguyen, Toan Tran, Yarin Gal, Atılım Güneş Baydin

Domain generalization refers to the problem where we aim to train a model on data from a set of source domains so that the model can generalize to unseen target domains.

Domain Generalization Representation Learning

On Statistical Bias In Active Learning: How and When To Fix It

no code implementations ICLR 2021 Sebastian Farquhar, Yarin Gal, Tom Rainforth

Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution.

Active Learning

Technology Readiness Levels for Machine Learning Systems

no code implementations11 Jan 2021 Alexander Lavin, Ciarán M. Gilligan-Lee, Alessya Visnjic, Siddha Ganju, Dava Newman, Atılım Güneş Baydin, Sujoy Ganguly, Danny Lange, Amit Sharma, Stephan Zheng, Eric P. Xing, Adam Gibson, James Parr, Chris Mattmann, Yarin Gal

The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end.

PowerEvaluationBALD: Efficient Evaluation-Oriented Deep (Bayesian) Active Learning with Stochastic Acquisition Functions

no code implementations10 Jan 2021 Andreas Kirsch, Yarin Gal

We develop BatchEvaluationBALD, a new acquisition function for deep Bayesian active learning, as an expansion of BatchBALD that takes into account an evaluation set of unlabeled data, for example, the pool set.

Active Learning

Unpacking Information Bottlenecks: Surrogate Objectives for Deep Learning

no code implementations1 Jan 2021 Andreas Kirsch, Clare Lyle, Yarin Gal

The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models.

Density Estimation

Invariant Representations for Reinforcement Learning without Reconstruction

no code implementations ICLR 2021 Amy Zhang, Rowan Thomas McAllister, Roberto Calandra, Yarin Gal, Sergey Levine

We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.

Causal Inference reinforcement-learning +1

Variational Deterministic Uncertainty Quantification

no code implementations1 Jan 2021 Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, Yarin Gal

Building on recent advances in uncertainty quantification using a single deep deterministic model (DUQ), we introduce variational Deterministic Uncertainty Quantification (vDUQ).

Causal Inference

Multi-Channel Auto-Calibration for the Atmospheric Imaging Assembly using Machine Learning

1 code implementation27 Dec 2020 Luiz F. G. dos Santos, Souvik Bose, Valentina Salvatelli, Brad Neuberg, Mark C. M. Cheung, Miho Janvier, Meng Jin, Yarin Gal, Paul Boerner, Atılım Güneş Baydin

Our approach establishes the framework for a novel technique to calibrate EUV instruments and advance our understanding of the cross-channel relation between different EUV channels.

Semi-supervised Learning of Galaxy Morphology using Equivariant Transformer Variational Autoencoders

no code implementations17 Nov 2020 Mizu Nishikawa-Toomey, Lewis Smith, Yarin Gal

We show that this novel architecture leads to improvements in accuracy when used for the galaxy morphology classification task on the Galaxy Zoo data set.

General Classification Morphology classification

Inter-domain Deep Gaussian Processes

no code implementations1 Nov 2020 Tim G. J. Rudner, Dino Sejdinovic, Yarin Gal

We propose Inter-domain Deep Gaussian Processes, an extension of inter-domain shallow GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs), and demonstrate how to leverage existing approximate inference methods to perform simple and scalable approximate inference using inter-domain features in DGPs.

Gaussian Processes

On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes

1 code implementation1 Nov 2020 Tim G. J. Rudner, Oscar Key, Yarin Gal, Tom Rainforth

We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) issues.

Gaussian Processes Variational Inference

A Bayesian Perspective on Training Speed and Model Selection

no code implementations NeurIPS 2020 Clare Lyle, Lisa Schut, Binxin Ru, Yarin Gal, Mark van der Wilk

This provides two major insights: first, that a measure of a model's training speed can be used to estimate its marginal likelihood.

Model Selection

Interlocking Backpropagation: Improving depthwise model-parallelism

1 code implementation8 Oct 2020 Aidan N. Gomez, Oscar Key, Kuba Perlin, Stephen Gou, Nick Frosst, Jeff Dean, Yarin Gal

Motivated by poor resource utilisation, we introduce a class of intermediary strategies between local and global learning referred to as interlocking backpropagation.

Image Classification

Revisiting the Train Loss: an Efficient Performance Estimator for Neural Architecture Search

no code implementations28 Sep 2020 Binxin Ru, Clare Lyle, Lisa Schut, Mark van der Wilk, Yarin Gal

Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS).

Model Selection Neural Architecture Search

Improving compute efficacy frontiers with SliceOut

no code implementations21 Jul 2020 Pascal Notin, Aidan N. Gomez, Joanna Yoo, Yarin Gal

Pushing forward the compute efficacy frontier in deep learning is critical for tasks that require frequent model re-training or workloads that entail training a large number of models.

Identifying Causal-Effect Inference Failure with Uncertainty-Aware Models

1 code implementation NeurIPS 2020 Andrew Jesson, Sören Mindermann, Uri Shalit, Yarin Gal

We show that our methods enable us to deal gracefully with situations of "no-overlap", common in high-dimensional data, where standard applications of causal effect approaches fail.

Single Shot Structured Pruning Before Training

no code implementations1 Jul 2020 Joost van Amersfoort, Milad Alizadeh, Sebastian Farquhar, Nicholas Lane, Yarin Gal

We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training.

Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?

2 code implementations ICML 2020 Angelos Filos, Panagiotis Tigas, Rowan Mcallister, Nicholas Rhinehart, Sergey Levine, Yarin Gal

Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment, typically leading to arbitrary deductions and poorly-informed decisions.

Autonomous Vehicles OOD Detection

Learning Invariant Representations for Reinforcement Learning without Reconstruction

2 code implementations18 Jun 2020 Amy Zhang, Rowan McAllister, Roberto Calandra, Yarin Gal, Sergey Levine

We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.

Causal Inference reinforcement-learning +1

Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers

no code implementations8 Jun 2020 Tim Z. Xiao, Aidan N. Gomez, Yarin Gal

We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models.

Machine Translation Translation

Speedy Performance Estimation for Neural Architecture Search

2 code implementations NeurIPS 2021 Binxin Ru, Clare Lyle, Lisa Schut, Miroslav Fil, Mark van der Wilk, Yarin Gal

Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS).

Model Selection Neural Architecture Search

Uncertainty Evaluation Metric for Brain Tumour Segmentation

no code implementations MIDL 2019 Raghav Mehta, Angelos Filos, Yarin Gal, Tal Arbel

In this paper, we develop a metric designed to assess and rank uncertainty measures for the task of brain tumour sub-tissue segmentation in the BraTS 2019 sub-challenge on uncertainty quantification.

BayesOpt Adversarial Attack

1 code implementation ICLR 2020 Binxin Ru, Adam Cobb, Arno Blaas, Yarin Gal

Black-box adversarial attacks require a large number of attempts before finding successful adversarial examples that are visually indistinguishable from the original input.

Adversarial Attack Bayesian Optimisation +2

On the Benefits of Invariance in Neural Networks

no code implementations1 May 2020 Clare Lyle, Mark van der Wilk, Marta Kwiatkowska, Yarin Gal, Benjamin Bloem-Reddy

Many real world data analysis problems exhibit invariant structure, and models that take advantage of this structure have shown impressive empirical performance, particularly in deep learning.

Data Augmentation

Capsule Networks -- A Probabilistic Perspective

no code implementations7 Apr 2020 Lewis Smith, Lisa Schut, Yarin Gal, Mark van der Wilk

'Capsule' models try to explicitly represent the poses of objects, enforcing a linear relationship between an object's pose and that of its constituent parts.

Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning

no code implementations27 Mar 2020 Andreas Kirsch, Clare Lyle, Yarin Gal

The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models.

Density Estimation

Invariant Causal Prediction for Block MDPs

1 code implementation ICML 2020 Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, Doina Precup

Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges.

Causal Inference Variable Selection

Uncertainty Estimation Using a Single Deep Deterministic Neural Network

2 code implementations4 Mar 2020 Joost van Amersfoort, Lewis Smith, Yee Whye Teh, Yarin Gal

We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass.

Out-of-Distribution Detection

Liberty or Depth: Deep Bayesian Neural Nets Do Not Need Complex Weight Posterior Approximations

no code implementations NeurIPS 2020 Sebastian Farquhar, Lewis Smith, Yarin Gal

We challenge the longstanding assumption that the mean-field approximation for variational inference in Bayesian neural networks is severely restrictive, and show this is not the case in deep networks.

Variational Inference

A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks

1 code implementation22 Dec 2019 Angelos Filos, Sebastian Farquhar, Aidan N. Gomez, Tim G. J. Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud de Kroon, Yarin Gal

From our comparison we conclude that some current techniques which solve benchmarks such as UCI `overfit' their uncertainty to the dataset---when evaluated on our benchmark these underperform in comparison to simpler baselines.

Out-of-Distribution Detection

Adversarial recovery of agent rewards from latent spaces of the limit order book

no code implementations9 Dec 2019 Jacobo Roa-Vicens, Yuanbo Wang, Virgile Mison, Yarin Gal, Ricardo Silva

In this paper, we explore whether adversarial inverse RL algorithms can be adapted and trained within such latent space simulations from real market data, while maintaining their ability to recover agent rewards robust to variations in the underlying dynamics, and transfer them to new regimes of the original environment.

Using U-Nets to Create High-Fidelity Virtual Observations of the Solar Corona

1 code implementation10 Nov 2019 Valentina Salvatelli, Souvik Bose, Brad Neuberg, Luiz F. G. dos Santos, Mark Cheung, Miho Janvier, Atilim Gunes Baydin, Yarin Gal, Meng Jin

The synergy between machine learning and this enormous amount of data has the potential, still largely unexploited, to advance our understanding of the Sun and extend the capabilities of heliophysics missions.

Translation

Auto-Calibration of Remote Sensing Solar Telescopes with Deep Learning

1 code implementation10 Nov 2019 Brad Neuberg, Souvik Bose, Valentina Salvatelli, Luiz F. G. dos Santos, Mark Cheung, Miho Janvier, Atilim Gunes Baydin, Yarin Gal, Meng Jin

As a part of NASA's Heliophysics System Observatory (HSO) fleet of satellites, the Solar Dynamics Observatory (SDO) has continuously monitored the Sun since2010.

Machine Learning for Generalizable Prediction of Flood Susceptibility

no code implementations15 Oct 2019 Chelsea Sidrane, Dylan J Fitzpatrick, Andrew Annex, Diane O'Donoghue, Yarin Gal, Piotr Biliński

In this work, we develop generalizable, multi-basin models of river flooding susceptibility using geographically-distributed data from the USGS stream gauge network.

Correlation of Auroral Dynamics and GNSS Scintillation with an Autoencoder

no code implementations4 Oct 2019 Kara Lamb, Garima Malhotra, Athanasios Vlontzos, Edward Wagstaff, Atılım Günes Baydin, Anahita Bhiwandiwalla, Yarin Gal, Alfredo Kalaitzis, Anthony Reina, Asti Bhatt

High energy particles originating from solar activity travel along the the Earth's magnetic field and interact with the atmosphere around the higher latitudes.

Flood Detection On Low Cost Orbital Hardware

no code implementations4 Oct 2019 Gonzalo Mateo-Garcia, Silviu Oprea, Lewis Smith, Josh Veitch-Michaelis, Guy Schumann, Yarin Gal, Atılım Güneş Baydin, Dietmar Backes

Satellite imaging is a critical technology for monitoring and responding to natural disasters such as flooding.

Prediction of GNSS Phase Scintillations: A Machine Learning Approach

no code implementations3 Oct 2019 Kara Lamb, Garima Malhotra, Athanasios Vlontzos, Edward Wagstaff, Atılım Günes Baydin, Anahita Bhiwandiwalla, Yarin Gal, Alfredo Kalaitzis, Anthony Reina, Asti Bhatt

We propose a novel architecture and loss function to predict 1 hour in advance the magnitude of phase scintillations within a time window of plus-minus 5 minutes with state-of-the-art performance.

Model-based Saliency for the Detection of Adversarial Examples

no code implementations25 Sep 2019 Lisa Schut, Yarin Gal

Adversarial perturbations cause a shift in the salient features of an image, which may result in a misclassification.

Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control

no code implementations21 Sep 2019 Rhiannon Michelmore, Matthew Wicker, Luca Laurenti, Luca Cardelli, Yarin Gal, Marta Kwiatkowska

Deep neural network controllers for autonomous driving have recently benefited from significant performance improvements, and have begun deployment in the real world.

Autonomous Driving Bayesian Inference +1

Generalizing from a few environments in safety-critical reinforcement learning

1 code implementation2 Jul 2019 Zachary Kenton, Angelos Filos, Owain Evans, Yarin Gal

Before deploying autonomous agents in the real world, we need to be confident they will perform safely in novel situations.

reinforcement-learning

Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning

4 code implementations1 Jul 2019 Sebastian Farquhar, Michael Osborne, Yarin Gal

The Radial BNN is motivated by avoiding a sampling problem in 'mean-field' variational inference (MFVI) caused by the so-called 'soap-bubble' pathology of multivariate Gaussians.

Continual Learning Variational Inference

BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning

2 code implementations NeurIPS 2019 Andreas Kirsch, Joost van Amersfoort, Yarin Gal

We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning.

Active Learning

Towards Inverse Reinforcement Learning for Limit Order Book Dynamics

no code implementations11 Jun 2019 Jacobo Roa-Vicens, Cyrine Chtourou, Angelos Filos, Francisco Rullan, Yarin Gal, Ricardo Silva

Given the expert agent's demonstrations, we attempt to discover their strategy by modelling their latent reward function using linear and Gaussian process (GP) regressors from previous literature, and our own approach through Bayesian neural networks (BNN).

reinforcement-learning

Learning Sparse Networks Using Targeted Dropout

2 code implementations31 May 2019 Aidan N. Gomez, Ivan Zhang, Siddhartha Rao Kamalakara, Divyam Madaan, Kevin Swersky, Yarin Gal, Geoffrey E. Hinton

Before computing the gradients for each weight update, targeted dropout stochastically selects a set of units or weights to be dropped using a simple self-reinforcing sparsity criterion and then computes the gradients for the remaining weights.

Network Pruning Neural Network Compression

An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval

1 code implementation25 May 2019 Adam D. Cobb, Michael D. Himes, Frank Soboczenski, Simone Zorzan, Molly D. O'Beirne, Atılım Güneş Baydin, Yarin Gal, Shawn D. Domagal-Goldman, Giada N. Arney, Daniel Angerhausen

We expand upon their approach by presenting a new machine learning model, \texttt{plan-net}, based on an ensemble of Bayesian neural networks that yields more accurate inferences than the random forest for the same data set of synthetic transmission spectra.

Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning

1 code implementation17 May 2019 Mike Walmsley, Lewis Smith, Chris Lintott, Yarin Gal, Steven Bamford, Hugh Dickinson, Lucy Fortson, Sandor Kruk, Karen Masters, Claudia Scarlata, Brooke Simmons, Rebecca Smethurst, Darryl Wright

We use Bayesian convolutional neural networks and a novel generative model of Galaxy Zoo volunteer responses to infer posteriors for the visual morphology of galaxies.

Active Learning

An Empirical study of Binary Neural Networks' Optimisation

1 code implementation ICLR 2019 Milad Alizadeh, Javier Fernández-Marqués, Nicholas D. Lane, Yarin Gal

In this work, we empirically identify and study the effectiveness of the various ad-hoc techniques commonly used in the literature, providing best-practices for efficient training of binary models.

A Unifying Bayesian View of Continual Learning

2 code implementations18 Feb 2019 Sebastian Farquhar, Yarin Gal

From a Bayesian perspective, continual learning seems straightforward: Given the model posterior one would simply use this as the prior for the next task.

Continual Learning

Differentially Private Continual Learning

no code implementations18 Feb 2019 Sebastian Farquhar, Yarin Gal

Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons.

Continual Learning Variational Inference

Evaluating Bayesian Deep Learning Methods for Semantic Segmentation

1 code implementation30 Nov 2018 Jishnu Mukhoti, Yarin Gal

Deep learning has been revolutionary for computer vision and semantic segmentation in particular, with Bayesian Deep Learning (BDL) used to obtain uncertainty maps from deep models when predicting semantic classes.

Autonomous Driving Semantic Segmentation

On the Importance of Strong Baselines in Bayesian Deep Learning

1 code implementation23 Nov 2018 Jishnu Mukhoti, Pontus Stenetorp, Yarin Gal

Like all sub-fields of machine learning Bayesian Deep Learning is driven by empirical validation of its theoretical proposals.

Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control

no code implementations16 Nov 2018 Rhiannon Michelmore, Marta Kwiatkowska, Yarin Gal

A rise in popularity of Deep Neural Networks (DNNs), attributed to more powerful GPUs and widely available datasets, has seen them being increasingly used within safety-critical domains.

Autonomous Driving Self-Driving Cars

Bayesian Deep Learning for Exoplanet Atmospheric Retrieval

no code implementations8 Nov 2018 Frank Soboczenski, Michael D. Himes, Molly D. O'Beirne, Simone Zorzan, Atilim Gunes Baydin, Adam D. Cobb, Yarin Gal, Daniel Angerhausen, Massimo Mascaro, Giada N. Arney, Shawn D. Domagal-Goldman

Here we present an ML-based retrieval framework called Intelligent exoplaNet Atmospheric RetrievAl (INARA) that consists of a Bayesian deep learning model for retrieval and a data set of 3, 000, 000 synthetic rocky exoplanetary spectra generated using the NASA Planetary Spectrum Generator.

Targeted Dropout

1 code implementation NIPS Workshop CDNNRIA 2018 Aidan N. Gomez, Ivan Zhang, Kevin Swersky, Yarin Gal, Geoffrey E. Hinton

Neural networks are extremely flexible models due to their large number of parameters, which is beneficial for learning, but also highly redundant.

Towards Robust Evaluations of Continual Learning

no code implementations24 May 2018 Sebastian Farquhar, Yarin Gal

Experiments used in current continual learning research do not faithfully assess fundamental challenges of learning continually.

Continual Learning

Loss-Calibrated Approximate Inference in Bayesian Neural Networks

1 code implementation10 May 2018 Adam D. Cobb, Stephen J. Roberts, Yarin Gal

Current approaches in approximate inference for Bayesian neural networks minimise the Kullback-Leibler divergence to approximate the true posterior over the weights.

Autonomous Driving Semantic Segmentation

Understanding Measures of Uncertainty for Adversarial Example Detection

2 code implementations22 Mar 2018 Lewis Smith, Yarin Gal

Measuring uncertainty is a promising technique for detecting adversarial examples, crafted inputs on which the model predicts an incorrect class with high confidence.

General Classification

BRUNO: A Deep Recurrent Model for Exchangeable Data

3 code implementations NeurIPS 2018 Iryna Korshunova, Jonas Degrave, Ferenc Huszár, Yarin Gal, Arthur Gretton, Joni Dambre

We present a novel model architecture which leverages deep learning tools to perform exact Bayesian inference on sets of high dimensional, complex observations.

Anomaly Detection Bayesian Inference +2

Vprop: Variational Inference using RMSprop

no code implementations4 Dec 2017 Mohammad Emtiyaz Khan, Zuozhu Liu, Voot Tangkaratt, Yarin Gal

Overall, this paper presents Vprop as a principled, computationally-efficient, and easy-to-implement method for Bayesian deep learning.

Variational Inference

Concrete Dropout

4 code implementations NeurIPS 2017 Yarin Gal, Jiri Hron, Alex Kendall

Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks.

reinforcement-learning

Real Time Image Saliency for Black Box Classifiers

3 code implementations NeurIPS 2017 Piotr Dabkowski, Yarin Gal

In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier.

Saliency Detection

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

9 code implementations NeurIPS 2017 Alex Kendall, Yarin Gal

On the other hand, epistemic uncertainty accounts for uncertainty in the model -- uncertainty which can be explained away given enough data.

Depth Estimation Semantic Segmentation

Deep Bayesian Active Learning with Image Data

3 code implementations ICML 2017 Yarin Gal, Riashat Islam, Zoubin Ghahramani

In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way.

Active Learning

Dropout Inference in Bayesian Neural Networks with Alpha-divergences

1 code implementation ICML 2017 Yingzhen Li, Yarin Gal

To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed.

Variational Inference

A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

15 code implementations NeurIPS 2016 Yarin Gal, Zoubin Ghahramani

Recent results at the intersection of Bayesian modelling and deep learning offer a Bayesian interpretation of common deep learning techniques such as dropout.

Bayesian Inference Language Modelling +2

Dirichlet Fragmentation Processes

no code implementations16 Sep 2015 Hong Ge, Yarin Gal, Zoubin Ghahramani

In this paper, first we review the theory of random fragmentation processes [Bertoin, 2006], and a number of existing methods for modelling trees, including the popular nested Chinese restaurant process (nCRP).

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

22 code implementations6 Jun 2015 Yarin Gal, Zoubin Ghahramani

In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computational cost.

Bayesian Inference Gaussian Processes +1

Dropout as a Bayesian Approximation: Appendix

1 code implementation6 Jun 2015 Yarin Gal, Zoubin Ghahramani

We show that a neural network with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to a well known Bayesian model.

Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs

1 code implementation9 Mar 2015 Yarin Gal, Richard Turner

We model the covariance function with a finite Fourier series approximation and treat it as a random variable.

Variational Inference

Latent Gaussian Processes for Distribution Estimation of Multivariate Categorical Data

1 code implementation7 Mar 2015 Yarin Gal, Yutian Chen, Zoubin Ghahramani

Building on these ideas we propose a Bayesian model for the unsupervised task of distribution estimation of multivariate categorical data.

Gaussian Processes Imputation +1

Semantics, Modelling, and the Problem of Representation of Meaning -- a Brief Survey of Recent Literature

no code implementations28 Feb 2014 Yarin Gal

Over the past 50 years many have debated what representation should be used to capture the meaning of natural language utterances.

Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models - a Gentle Tutorial

no code implementations6 Feb 2014 Yarin Gal, Mark van der Wilk

In this tutorial we explain the inference procedures developed for the sparse Gaussian process (GP) regression and Gaussian process latent variable model (GPLVM).

Gaussian Processes Variational Inference

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

1 code implementation NeurIPS 2014 Yarin Gal, Mark van der Wilk, Carl E. Rasmussen

We show that GP performance improves with increasing amounts of data in regression (on flight data with 2 million records) and latent variable modelling (on MNIST).

Dimensionality Reduction Gaussian Processes +1

Cannot find the paper you are looking for? You can Submit a new open access paper.