Search Results for author:
Found 169 papers, 91 papers with code
Inter-domain Deep Gaussian Processes with RKHS Fourier Features
We propose Inter-domain Deep Gaussian Processes with RKHS Fourier Features, an extension of shallow inter-domain GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs) and demonstrate how to leverage existing approximate inference approaches to perform simple and scalable approximate inference on Inter-domain Deep Gaussian Processes.
Simple and Scalable Epistemic Uncertainty Estimation Using a Single Deep Deterministic Neural Network
We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass.
TextCAVs: Debugging vision models using text
Concept-based interpretability methods are a popular form of explanation for deep learning models which provide explanations in the form of high-level human interpretable concepts.
Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks
In this work, we investigate the impact of weight space permutation symmetries on VI.
Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
We propose semantic entropy probes (SEPs), a cheap and reliable method for uncertainty quantification in Large Language Models (LLMs).
The Benefits and Risks of Transductive Approaches for AI Fairness
Recently, transductive learning methods, which leverage holdout sets during training, have gained popularity for their potential to improve speed, accuracy, and fairness in machine learning models.
Deep Bayesian Active Learning for Preference Modeling in Large Language Models
We address this by proposing the Bayesian Active Learner for Preference Modeling (BAL-PM), a novel stochastic acquisition policy that not only targets points of high epistemic uncertainty according to the preference model but also seeks to maximize the entropy of the acquired prompt distribution in the feature space spanned by the employed LLM.
Estimating the Hallucination Rate of Generative AI
We develop a new method that takes an ICL problem -- that is, a CGM, a dataset, and a prediction question -- and estimates the probability that a CGM will generate a hallucination.
Challenges and Considerations in the Evaluation of Bayesian Causal Discovery
Unlike non-Bayesian causal discovery, which relies on a single estimated causal graph and model parameters for assessment, evaluating BCD presents challenges due to the nature of its inferred quantity - the posterior distribution.
Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
To address this problem, we propose Kernel Language Entropy (KLE), a novel method for uncertainty estimation in white- and black-box LLMs.
Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control
This has led to the emergence of pre-trained vision-language models as a tool for transferring representations learned from internet-scale data to downstream tasks and new domains.
Explaining Explainability: Understanding Concept Activation Vectors
We introduce tools designed to detect the presence of these properties, provide insight into how they affect the derived explanations, and provide recommendations to minimise their impact.
Bayesian Preference Elicitation with Language Models
We introduce OPEN (Optimal Preference Elicitation with Natural language) a framework that uses BOED to guide the choice of informative questions and an LM to extract features and translate abstract BOED queries into natural language questions.
Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?
To answer this question, we consider a set of tailored offline reinforcement learning datasets that exhibit causal ambiguity and assess the ability of active sampling techniques to reduce causal confusion at evaluation.
Tractable Function-Space Variational Inference in Bayesian Neural Networks
Recognizing that the primary object of interest in most settings is the distribution over functions induced by the posterior distribution over neural network parameters, we frame Bayesian inference in neural networks explicitly as inferring a posterior distribution over functions and propose a scalable function-space variational inference method that allows incorporating prior information and results in reliable predictive uncertainty estimates.
Continual Learning via Sequential Function-Space Variational Inference
Sequential Bayesian inference over predictive functions is a natural framework for continual learning from streams of data.
DiscoBAX: Discovery of Optimal Intervention Sets in Genomic Experiment Design
The discovery of therapeutics to treat genetically-driven pathologies relies on identifying genes involved in the underlying disease mechanisms.
Form follows Function: Text-to-Text Conditional Graph Generation based on Functional Requirements
This work focuses on the novel problem setting of generating graphs conditioned on a description of the graph's functional requirements in a downstream task.
How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions
Large language models (LLMs) can "lie", which we define as outputting false statements despite "knowing" the truth in a demonstrable sense.
Fine-tuning can cripple your foundation model; preserving features may be the solution
This is an undesirable effect of fine-tuning as a substantial amount of resources was used to learn these pre-trained concepts in the first place.
In-Context Learning Learns Label Relationships but Is Not Conventional Learning
The predictions of Large Language Models (LLMs) on downstream tasks often improve significantly when including examples of the input--label relationship in the context.
LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?
Specifically, we demonstrate that semantic censorship can be perceived as an undecidable problem, highlighting the inherent challenges in censorship that arise due to LLMs' programmatic and instruction-following capabilities.
BatchGFN: Generative Flow Networks for Batch Active Learning
We introduce BatchGFN -- a novel approach for pool-based active learning that uses generative flow networks to sample sets of data points proportional to a batch reward.
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages
We show that the additive term is bounded proportional to the Lipschitz constant of the value function, which offers theoretical grounding for spectral normalization of critic weights.
The Curse of Recursion: Training on Generated Data Makes Models Forget
It is now clear that large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images.
Prediction-Oriented Bayesian Active Learning
Information-theoretic approaches to active learning have traditionally focused on maximising the information gathered about the model parameters, most commonly by optimising the BALD score.
Revisiting Automated Prompting: Are We Actually Doing Better?
Current literature demonstrates that Large Language Models (LLMs) are great few-shot learners, and prompting significantly increases their performance on a range of downstream tasks in a few-shot learning setting.
Differentiable Multi-Target Causal Bayesian Experimental Design
We introduce a gradient-based approach for the problem of Bayesian optimal experimental design to learn causal models in a batch setting -- a critical component for causal discovery from finite data where interventions can be costly or risky.
Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
We introduce a method to measure uncertainty in large language models.
Deep Deterministic Uncertainty: A New Simple Baseline
Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive.
On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations
KL-regularized reinforcement learning from expert demonstrations has proved successful in improving the sample efficiency of deep reinforcement learning algorithms, allowing them to be applied to challenging physical real-world tasks.
CLAM: Selective Clarification for Ambiguous Questions with Generative Language Models
Users often ask dialogue systems ambiguous questions that require clarification.
Using uncertainty-aware machine learning models to study aerosol-cloud interactions
Aerosol-cloud interactions (ACI) include various effects that result from aerosols entering a cloud, and affecting cloud properties.
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks
We use these tasks to benchmark well-established and state-of-the-art Bayesian deep learning methods on task-specific evaluation metrics.
Discovering Long-period Exoplanets using Deep Learning with Citizen Science Labels
Automated planetary transit detection has become vital to prioritize candidates for expert analysis given the scale of modern telescopic surveys.
Exploring Low Rank Training of Deep Neural Networks
Training deep neural networks in low rank, i. e. with factorised layers, is of particular interest to the community: it offers efficiency over unfactorised training in terms of both memory consumption and training time.
Exploring the Limits of Synthetic Creation of Solar EUV Images via Image-to-Image Translation
The Solar Dynamics Observatory (SDO), a NASA multi-spectral decade-long mission that has been daily producing terabytes of observational data from the Sun, has been recently used as a use-case to demonstrate the potential of machine learning methodologies and to pave the way for future deep-space mission planning.
Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantities
Recently proposed methods in data subset selection, that is active learning and active sampling, use Fisher information, Hessians, similarity matrices based on gradients, and gradient lengths to estimate how informative data is for a model's training.
Plex: Towards Reliability using Pretrained Large Model Extensions
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures.
Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt
But most computation and time is wasted on redundant and noisy points that are already learnt or not learnable.
Learning Dynamics and Generalization in Reinforcement Learning
Solving a reinforcement learning (RL) problem poses two competing challenges: fitting a potentially discontinuous value function, and generalizing well to new observations.
Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval
The ability to accurately model the fitness landscape of protein sequences is critical to a wide range of applications, from quantifying the effects of human variants on disease likelihood, to predicting immune-escape mutations in viruses and designing novel biotherapeutic proteins.
Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Sampling
They are more realistic than previously suggested ones, building on work by Wen et al. (2021) and Osband et al. (2022), and focus on evaluating the performance of approximate BNNs in an online supervised setting.
Global geomagnetic perturbation forecasting using Deep Learning
The model summarizes 2 hours of solar wind measurement using a Gated Recurrent Unit, and generates forecasts of coefficients which are folded with a spherical harmonic basis to enable global forecasts.
Scalable Sensitivity and Uncertainty Analysis for Causal-Effect Estimates of Continuous-Valued Interventions
Estimating the effects of continuous-valued interventions from observational data is a critically important task for climate science, healthcare, and economics.
Interventions, Where and How? Experimental Design for Causal Models at Scale
Existing methods in experimental design for causal discovery from limited data either rely on linear assumptions for the SCM or select only the intervention target.
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients
Pruning neural networks at initialization would enable us to find sparse models that retain the accuracy of the original network while consuming fewer computational resources for training and inference.
Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation.
A Note on "Assessing Generalization of SGD via Disagreement"
Several recent works find empirically that the average test error of deep neural networks can be estimated via the prediction disagreement of models, which does not require labels.
DARTS without a Validation Set: Optimizing the Marginal Likelihood
The success of neural architecture search (NAS) has historically been limited by excessive compute requirements.
QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results
In this study, we explore and evaluate a score developed during the BraTS 2019 and BraTS 2020 task on uncertainty quantification (QU-BraTS) and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation.
Decomposing Representations for Deterministic Uncertainty Estimation
Uncertainty estimation is a key component in any deployed machine learning system.
DeDUCE: Generating Counterfactual Explanations Efficiently
This is the aim of algorithms generating counterfactual explanations.
Contrastive Representation Learning with Trainable Augmentation Channel
In contrastive representation learning, data representation is trained so that it can classify the image instances even when the images are altered by augmentations.
Multi-Spectral Multi-Image Super-Resolution of Sentinel-2 with Radiometric Consistency Losses and Its Effect on Building Delineation
High resolution remote sensing imagery is used in broad range of tasks, including detection and classification of objects.
Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data
We introduce causal, Bayesian acquisition functions grounded in information theory that bias data acquisition towards regions with overlapping support to maximize sample efficiency for learning personalized treatment effects.
Deep Deterministic Uncertainty for Semantic Segmentation
We extend Deep Deterministic Uncertainty (DDU), a method for uncertainty estimation using feature space densities, to semantic segmentation.
Using Non-Linear Causal Models to Study Aerosol-Cloud Interactions in the Southeast Pacific
Aerosol-cloud interactions include a myriad of effects that all begin when aerosol enters a cloud and acts as cloud condensation nuclei (CCN).
GeneDisco: A Benchmark for Experimental Design in Drug Discovery
GeneDisco contains a curated set of multiple publicly available experimental data sets as well as open-source implementations of state-of-the-art active learning policies for experimental design and exploration.
Quantifying Uncertainty for Machine Learning Based Diagnostic
Virtual Diagnostic (VD) is a deep learning tool that can be used to predict a diagnostic output.
Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
However, many tasks of practical interest have different modalities, such as tabular data, audio, text, or sensor data, which offer significant challenges involving regression and discrete or continuous structured prediction.
Ranked #2 on
Weather Forecasting
on Shifts
Prioritized training on points that are learnable, worth learning, and not yet learned (workshop version)
We introduce Goldilocks Selection, a technique for faster model training which selects a sequence of training points that are "just right".
Improving black-box optimization in VAE latent space using decoder uncertainty
Optimization in the latent space of variational autoencoders is a promising approach to generate high-dimensional discrete objects that maximize an expensive black-box property (e. g., drug-likeness in molecular generation, function approximation with arithmetic expressions).
A Practical & Unified Notation for Information-Theoretic Quantities in ML
A practical notation can convey valuable intuitions and concisely express new ideas.
Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
Expanding on MacKay (1992), we argue that conventional model-based methods for active learning - like BALD - have a fundamental shortfall: they fail to directly account for the test-time distribution of the input variables.
Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning
We examine a simple stochastic strategy for adapting well-known single-point acquisition functions to allow batch active learning.
KL Guided Domain Adaptation
A common approach in the domain adaptation literature is to learn a representation of the input that has the same (marginal) distribution over the source and the target domain.
Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
In this paper we introduce Uncertainty Baselines: high-quality implementations of standard and state-of-the-art deep learning methods on a variety of tasks.
Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning
We challenge a common assumption underlying most supervised deep learning: that a model makes a prediction depending only on its parameters and the features of a single input.
Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective
ResNets constrained to be bi-Lipschitz, that is, approximately distance preserving, have been a crucial component of recently proposed techniques for deterministic uncertainty quantification in neural models.
Outcome-Driven Reinforcement Learning via Variational Inference
While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it.
Physically-Consistent Generative Adversarial Networks for Coastal Flood Visualization
Our work aims to enable more visual communication of large-scale climate impacts via visualizing the output of coastal flood models as satellite imagery.
Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties
Counterfactual explanations (CEs) are a practical tool for demonstrating why machine learning classifiers make particular decisions.
Robustness to Pruning Predicts Generalization in Deep Neural Networks
Existing generalization measures that aim to capture a model's simplicity based on parameter counts or norms fail to explain generalization in overparameterized deep neural networks.
Resolving Causal Confusion in Reinforcement Learning via Robust Exploration
To address this, we present a robust exploration strategy which enables causal hypothesis-testing by interaction with the environment.
Active Testing: Sample-Efficient Model Evaluation
While approaches like active learning reduce the number of labels needed for model training, existing literature largely ignores the cost of labeling test data, typically unrealistically assuming large test sets for model evaluation.
Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding
We study the problem of learning conditional average treatment effects (CATE) from high-dimensional, observational data with unobserved confounders.
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
This allows us to disentangle shared features and dynamics of the environment from agent-specific rewards and policies.
Deep Deterministic Uncertainty: A Simple Baseline
Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive.
On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
Inducing point Gaussian process approximations are often considered a gold standard in uncertainty estimation since they retain many of the properties of the exact GP and scale to large datasets.
Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies
All classifications are used to train an ensemble of Bayesian convolutional neural networks (a state-of-the-art deep learning method) to predict posteriors for the detailed morphology of all 314, 000 galaxies.
Domain Invariant Representation Learning with Domain Density Transformations
Domain generalization refers to the problem where we aim to train a model on data from a set of source domains so that the model can generalize to unseen target domains.
Global Earth Magnetic Field Modeling and Forecasting with Spherical Harmonics Decomposition
Modeling and forecasting the solar wind-driven global magnetic field perturbations is an open challenge.
On Statistical Bias In Active Learning: How and When To Fix It
Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution.
Technology Readiness Levels for Machine Learning Systems
The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end.
PowerEvaluationBALD: Efficient Evaluation-Oriented Deep (Bayesian) Active Learning with Stochastic Acquisition Functions
We develop BatchEvaluationBALD, a new acquisition function for deep Bayesian active learning, as an expansion of BatchBALD that takes into account an evaluation set of unlabeled data, for example, the pool set.
Unpacking Information Bottlenecks: Surrogate Objectives for Deep Learning
The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models.
Variational Deterministic Uncertainty Quantification
Building on recent advances in uncertainty quantification using a single deep deterministic model (DUQ), we introduce variational Deterministic Uncertainty Quantification (vDUQ).
Invariant Representations for Reinforcement Learning without Reconstruction
We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.
Multi-Channel Auto-Calibration for the Atmospheric Imaging Assembly using Machine Learning
Our approach establishes the framework for a novel technique to calibrate EUV instruments and advance our understanding of the cross-channel relation between different EUV channels.
On Batch Normalisation for Approximate Bayesian Inference
We study batch normalisation in the context of variational inference methods in Bayesian neural networks, such as mean-field or MC Dropout.
Rethinking Function-Space Variational Inference in Bayesian Neural Networks
Bayesian neural networks (BNNs) define distributions over functions induced by distributions over parameters.
Semi-supervised Learning of Galaxy Morphology using Equivariant Transformer Variational Autoencoders
We show that this novel architecture leads to improvements in accuracy when used for the galaxy morphology classification task on the Galaxy Zoo data set.
Inter-domain Deep Gaussian Processes
We propose Inter-domain Deep Gaussian Processes, an extension of inter-domain shallow GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs), and demonstrate how to leverage existing approximate inference methods to perform simple and scalable approximate inference using inter-domain features in DGPs.
On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes
We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) issues.
A Bayesian Perspective on Training Speed and Model Selection
This provides two major insights: first, that a measure of a model's training speed can be used to estimate its marginal likelihood.
Physics-informed GANs for Coastal Flood Visualization
As climate change increases the intensity of natural disasters, society needs better tools for adaptation.
Interlocking Backpropagation: Improving depthwise model-parallelism
Motivated by poor resource utilisation in the global setting and poor task performance in the local setting, we introduce a class of intermediary strategies between local and global learning referred to as interlocking backpropagation.
Revisiting the Train Loss: an Efficient Performance Estimator for Neural Architecture Search
Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS).
How Robust are the Estimated Effects of Nonpharmaceutical Interventions against COVID-19?
To what extent are effectiveness estimates of nonpharmaceutical interventions (NPIs) against COVID-19 influenced by the assumptions our models make?
Improving compute efficacy frontiers with SliceOut
Pushing forward the compute efficacy frontier in deep learning is critical for tasks that require frequent model re-training or workloads that entail training a large number of models.
Identifying Causal-Effect Inference Failure with Uncertainty-Aware Models
We show that our methods enable us to deal gracefully with situations of "no-overlap", common in high-dimensional data, where standard applications of causal effect approaches fail.
Single Shot Structured Pruning Before Training
We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training.
Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?
Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment, typically leading to arbitrary deductions and poorly-informed decisions.
Learning Invariant Representations for Reinforcement Learning without Reconstruction
We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.
Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models.
Speedy Performance Estimation for Neural Architecture Search
Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS).
Uncertainty Evaluation Metric for Brain Tumour Segmentation
In this paper, we develop a metric designed to assess and rank uncertainty measures for the task of brain tumour sub-tissue segmentation in the BraTS 2019 sub-challenge on uncertainty quantification.
BayesOpt Adversarial Attack
Black-box adversarial attacks require a large number of attempts before finding successful adversarial examples that are visually indistinguishable from the original input.
On the Benefits of Invariance in Neural Networks
Many real world data analysis problems exhibit invariant structure, and models that take advantage of this structure have shown impressive empirical performance, particularly in deep learning.
Capsule Networks -- A Probabilistic Perspective
'Capsule' models try to explicitly represent the poses of objects, enforcing a linear relationship between an object's pose and that of its constituent parts.
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning
The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models.
Baryons from Mesons: A Machine Learning Perspective
Quantum chromodynamics (QCD) is the theory of the strong interaction.
Invariant Causal Prediction for Block MDPs
Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges.
Uncertainty Estimation Using a Single Deep Deterministic Neural Network
We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass.
Liberty or Depth: Deep Bayesian Neural Nets Do Not Need Complex Weight Posterior Approximations
We challenge the longstanding assumption that the mean-field approximation for variational inference in Bayesian neural networks is severely restrictive, and show this is not the case in deep networks.
A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks
From our comparison we conclude that some current techniques which solve benchmarks such as UCI `overfit' their uncertainty to the dataset---when evaluated on our benchmark these underperform in comparison to simpler baselines.
Adversarial recovery of agent rewards from latent spaces of the limit order book
In this paper, we explore whether adversarial inverse RL algorithms can be adapted and trained within such latent space simulations from real market data, while maintaining their ability to recover agent rewards robust to variations in the underlying dynamics, and transfer them to new regimes of the original environment.
Using U-Nets to Create High-Fidelity Virtual Observations of the Solar Corona
The synergy between machine learning and this enormous amount of data has the potential, still largely unexploited, to advance our understanding of the Sun and extend the capabilities of heliophysics missions.
Auto-Calibration of Remote Sensing Solar Telescopes with Deep Learning
As a part of NASA's Heliophysics System Observatory (HSO) fleet of satellites, the Solar Dynamics Observatory (SDO) has continuously monitored the Sun since2010.
Single-Frame Super-Resolution of Solar Magnetograms: Investigating Physics-Based Metrics \& Losses
Breakthroughs in our understanding of physical phenomena have traditionally followed improvements in instrumentation.
Probabilistic Super-Resolution of Solar Magnetograms: Generating Many Explanations and Measuring Uncertainties
Machine learning techniques have been successfully applied to super-resolution tasks on natural images where visually pleasing results are sufficient.
VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Trading off exploration and exploitation in an unknown environment is key to maximising expected return during learning.
Machine Learning for Generalizable Prediction of Flood Susceptibility
In this work, we develop generalizable, multi-basin models of river flooding susceptibility using geographically-distributed data from the USGS stream gauge network.
Flood Detection On Low Cost Orbital Hardware
Satellite imaging is a critical technology for monitoring and responding to natural disasters such as flooding.
Correlation of Auroral Dynamics and GNSS Scintillation with an Autoencoder
High energy particles originating from solar activity travel along the the Earth's magnetic field and interact with the atmosphere around the higher latitudes.
Prediction of GNSS Phase Scintillations: A Machine Learning Approach
We propose a novel architecture and loss function to predict 1 hour in advance the magnitude of phase scintillations within a time window of plus-minus 5 minutes with state-of-the-art performance.
Model-based Saliency for the Detection of Adversarial Examples
Adversarial perturbations cause a shift in the salient features of an image, which may result in a misclassification.
Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control
Deep neural network controllers for autonomous driving have recently benefited from significant performance improvements, and have begun deployment in the real world.
Generalizing from a few environments in safety-critical reinforcement learning
Before deploying autonomous agents in the real world, we need to be confident they will perform safely in novel situations.
Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
The Radial BNN is motivated by avoiding a sampling problem in 'mean-field' variational inference (MFVI) caused by the so-called 'soap-bubble' pathology of multivariate Gaussians.
BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning.
Towards Inverse Reinforcement Learning for Limit Order Book Dynamics
Given the expert agent's demonstrations, we attempt to discover their strategy by modelling their latent reward function using linear and Gaussian process (GP) regressors from previous literature, and our own approach through Bayesian neural networks (BNN).
Learning Sparse Networks Using Targeted Dropout
Before computing the gradients for each weight update, targeted dropout stochastically selects a set of units or weights to be dropped using a simple self-reinforcing sparsity criterion and then computes the gradients for the remaining weights.
An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval
We expand upon their approach by presenting a new machine learning model, \texttt{plan-net}, based on an ensemble of Bayesian neural networks that yields more accurate inferences than the random forest for the same data set of synthetic transmission spectra.
Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning
We use Bayesian convolutional neural networks and a novel generative model of Galaxy Zoo volunteer responses to infer posteriors for the visual morphology of galaxies.
An Empirical study of Binary Neural Networks' Optimisation
In this work, we empirically identify and study the effectiveness of the various ad-hoc techniques commonly used in the literature, providing best-practices for efficient training of binary models.
Sufficient Conditions for Robustness to Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
Lastly, we demonstrate the defence on a cats-vs-dogs image classification task with a VGG13 variant.
Differentially Private Continual Learning
Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons.
A Unifying Bayesian View of Continual Learning
From a Bayesian perspective, continual learning seems straightforward: Given the model posterior one would simply use this as the prior for the next task.
Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
Deep learning has been revolutionary for computer vision and semantic segmentation in particular, with Bayesian Deep Learning (BDL) used to obtain uncertainty maps from deep models when predicting semantic classes.
Ranked #8 on
Anomaly Detection
on Fishyscapes
On the Importance of Strong Baselines in Bayesian Deep Learning
Like all sub-fields of machine learning Bayesian Deep Learning is driven by empirical validation of its theoretical proposals.
Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control
A rise in popularity of Deep Neural Networks (DNNs), attributed to more powerful GPUs and widely available datasets, has seen them being increasingly used within safety-critical domains.
Bayesian Deep Learning for Exoplanet Atmospheric Retrieval
Here we present an ML-based retrieval framework called Intelligent exoplaNet Atmospheric RetrievAl (INARA) that consists of a Bayesian deep learning model for retrieval and a data set of 3, 000, 000 synthetic rocky exoplanetary spectra generated using the NASA Planetary Spectrum Generator.
Targeted Dropout
Neural networks are extremely flexible models due to their large number of parameters, which is beneficial for learning, but also highly redundant.
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
Uncertainty computation in deep learning is essential to design robust and reliable systems.
Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
Lastly, we demonstrate the defence on a cats-vs-dogs image classification task with a VGG13 variant.
Towards Robust Evaluations of Continual Learning
Experiments used in current continual learning research do not faithfully assess fundamental challenges of learning continually.
Loss-Calibrated Approximate Inference in Bayesian Neural Networks
Current approaches in approximate inference for Bayesian neural networks minimise the Kullback-Leibler divergence to approximate the true posterior over the weights.
Understanding Measures of Uncertainty for Adversarial Example Detection
Measuring uncertainty is a promising technique for detecting adversarial examples, crafted inputs on which the model predicts an incorrect class with high confidence.
BRUNO: A Deep Recurrent Model for Exchangeable Data
We present a novel model architecture which leverages deep learning tools to perform exact Bayesian inference on sets of high dimensional, complex observations.
Vprop: Variational Inference using RMSprop
Overall, this paper presents Vprop as a principled, computationally-efficient, and easy-to-implement method for Bayesian deep learning.
Concrete Dropout
Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks.
Real Time Image Saliency for Black Box Classifiers
In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier.
Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Numerous deep learning applications benefit from multi-task learning with multiple regression and classification objectives.
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
On the other hand, epistemic uncertainty accounts for uncertainty in the model -- uncertainty which can be explained away given enough data.
Ranked #106 on
Semantic Segmentation
on NYU Depth v2
Dropout Inference in Bayesian Neural Networks with Alpha-divergences
To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed.
Deep Bayesian Active Learning with Image Data
In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way.
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
Recent results at the intersection of Bayesian modelling and deep learning offer a Bayesian interpretation of common deep learning techniques such as dropout.
Ranked #35 on
Language Modelling
on Penn Treebank (Word Level)
Dirichlet Fragmentation Processes
In this paper, first we review the theory of random fragmentation processes [Bertoin, 2006], and a number of existing methods for modelling trees, including the popular nested Chinese restaurant process (nCRP).
Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
Convolutional neural networks (CNNs) work well on large datasets.