Search Results for author: Michael A. Osborne

Found 57 papers, 27 papers with code

A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Lifting

1 code implementation18 Apr 2024 Masaki Adachi, Satoshi Hayakawa, Martin Jørgensen, Saad Hamid, Harald Oberhauser, Michael A. Osborne

Parallelisation in Bayesian optimisation is a common strategy but faces several challenges: the need for flexibility in acquisition functions and kernel choices, flexibility dealing with discrete and continuous variables simultaneously, model misspecification, and lastly fast massive parallelisation.

Beyond Lengthscales: No-regret Bayesian Optimisation With Unknown Hyperparameters Of Any Type

no code implementations2 Feb 2024 Juliusz Ziomek, Masaki Adachi, Michael A. Osborne

Previously proposed algorithms with the no-regret property were only able to handle the special case of unknown lengthscales, reproducing kernel Hilbert space norm and applied only to the frequentist case.

Bayesian Optimisation

On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations

1 code implementation NeurIPS 2021 Tim G. J. Rudner, Cong Lu, Michael A. Osborne, Yarin Gal, Yee Whye Teh

KL-regularized reinforcement learning from expert demonstrations has proved successful in improving the sample efficiency of deep reinforcement learning algorithms, allowing them to be applied to challenging physical real-world tasks.

reinforcement-learning Reinforcement Learning (RL)

Bayesian Model Selection of Lithium-Ion Battery Models via Bayesian Quadrature

1 code implementation28 Oct 2022 Masaki Adachi, Yannick Kuhn, Birger Horstmann, Arnulf Latz, Michael A. Osborne, David A. Howey

We show that popular model selection criteria, such as root-mean-square error and Bayesian information criterion, can fail to select a parsimonious model in the case of a multimodal posterior.

Model Selection

Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization

2 code implementations18 Oct 2022 Samuel Daulton, Xingchen Wan, David Eriksson, Maximilian Balandat, Michael A. Osborne, Eytan Bakshy

We prove that under suitable reparameterizations, the BO policy that maximizes the probabilistic objective is the same as that which maximizes the AF, and therefore, PR enjoys the same regret bounds as the original BO policy using the underlying AF.

Bayesian Optimization

Log-Linear-Time Gaussian Processes Using Binary Tree Kernels

1 code implementation4 Oct 2022 Michael K. Cohen, Samuel Daulton, Michael A. Osborne

We present a new kernel that allows for Gaussian process regression in $O((n+m)\log(n+m))$ time.

Gaussian Processes regression

Bézier Gaussian Processes for Tall and Wide Data

no code implementations1 Sep 2022 Martin Jørgensen, Michael A. Osborne

We introduce a kernel that allows the number of summarising variables to grow exponentially with the number of input features, but requires only linear cost in both number of observations and input features.

Gaussian Processes

Bayesian Generational Population-Based Training

2 code implementations19 Jul 2022 Xingchen Wan, Cong Lu, Jack Parker-Holder, Philip J. Ball, Vu Nguyen, Binxin Ru, Michael A. Osborne

Leveraging the new highly parallelizable Brax physics engine, we show that these innovations lead to large performance gains, significantly outperforming the tuned baseline while learning entire configurations on the fly.

Bayesian Optimization Reinforcement Learning (RL)

Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombination

2 code implementations9 Jun 2022 Masaki Adachi, Satoshi Hayakawa, Martin Jørgensen, Harald Oberhauser, Michael A. Osborne

Empirically, we find that our approach significantly outperforms the sampling efficiency of both state-of-the-art BQ techniques and Nested Sampling in various real-world datasets, including lithium-ion battery analytics.

Bayesian Inference Numerical Integration

Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations

2 code implementations9 Jun 2022 Cong Lu, Philip J. Ball, Tim G. J. Rudner, Jack Parker-Holder, Michael A. Osborne, Yee Whye Teh

Using this suite of benchmarking tasks, we show that simple modifications to two popular vision-based online reinforcement learning algorithms, DreamerV2 and DrQ-v2, suffice to outperform existing offline RL methods and establish competitive baselines for continuous control in the visual domain.

Benchmarking Continuous Control +3

Robust Multi-Objective Bayesian Optimization Under Input Noise

1 code implementation15 Feb 2022 Samuel Daulton, Sait Cakmak, Maximilian Balandat, Michael A. Osborne, Enlu Zhou, Eytan Bakshy

In many manufacturing processes, the design parameters are subject to random input noise, resulting in a product that is often less performant than expected.

Bayesian Optimization

Adversarial Attacks on Graph Classification via Bayesian Optimisation

1 code implementation4 Nov 2021 Xingchen Wan, Henry Kenlay, Binxin Ru, Arno Blaas, Michael A. Osborne, Xiaowen Dong

While the majority of the literature focuses on such vulnerability in node-level classification tasks, little effort has been dedicated to analysing adversarial attacks on graph-level classification, an important problem with numerous real-life applications such as biochemistry and social network analysis.

Adversarial Robustness Bayesian Optimisation +1

Gaussian Process Sampling and Optimization with Approximate Upper and Lower Bounds

no code implementations22 Oct 2021 Vu Nguyen, Marc Peter Deisenroth, Michael A. Osborne

More specifically, we propose the first use of such bounds to improve Gaussian process (GP) posterior sampling and Bayesian optimization (BO).

Bayesian Optimization

Revisiting Design Choices in Offline Model-Based Reinforcement Learning

no code implementations8 Oct 2021 Cong Lu, Philip J. Ball, Jack Parker-Holder, Michael A. Osborne, Stephen J. Roberts

Significant progress has been made recently in offline model-based reinforcement learning, approaches which leverage a learned dynamics model.

Bayesian Optimization Model-based Reinforcement Learning +2

Universal Approximation of Functions on Sets

no code implementations5 Jul 2021 Edward Wagstaff, Fabian B. Fuchs, Martin Engelcke, Michael A. Osborne, Ingmar Posner

We provide a theoretical analysis of Deep Sets which shows that this universal approximation property is only guaranteed if the model's latent space is sufficiently high-dimensional.

Marginalising over Stationary Kernels with Bayesian Quadrature

no code implementations14 Jun 2021 Saad Hamid, Sebastian Schulze, Michael A. Osborne, Stephen J. Roberts

Marginalising over families of Gaussian Process kernels produces flexible model classes with well-calibrated uncertainty estimates.

Gaussian Process Bandit Optimization of the Thermodynamic Variational Objective

1 code implementation NeurIPS 2020 Vu Nguyen, Vaden Masrani, Rob Brekelmans, Michael A. Osborne, Frank Wood

Achieving the full promise of the Thermodynamic Variational Objective (TVO), a recently proposed variational lower bound on the log evidence involving a one-dimensional Riemann integral approximation, requires choosing a "schedule" of sorted discretization points.

Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search

1 code implementation13 Jun 2020 Vu Nguyen, Tam Le, Makoto Yamada, Michael A. Osborne

Building upon tree-Wasserstein (TW), which is a negative definite variant of OT, we develop a novel discrepancy for neural architectures, and demonstrate it within a Gaussian process surrogate model for the sequential NAS settings.

Neural Architecture Search

Bayesian Optimization for Iterative Learning

1 code implementation NeurIPS 2020 Vu Nguyen, Sebastian Schulze, Michael A. Osborne

We demonstrate the efficiency of our algorithm by tuning hyperparameters for the training of deep reinforcement learning agents and convolutional neural networks.

Bayesian Optimization reinforcement-learning +1

Adaptive Configuration Oracle for Online Portfolio Selection Methods

no code implementations22 Aug 2019 Favour M. Nyikosa, Michael A. Osborne, Stephen J. Roberts

Financial markets are complex environments that produce enormous amounts of noisy and non-stationary data.

Bayesian Optimization

Bayesian Optimisation over Multiple Continuous and Categorical Inputs

2 code implementations ICML 2020 Binxin Ru, Ahsan S. Alvi, Vu Nguyen, Michael A. Osborne, Stephen J. Roberts

Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges.

Bayesian Optimisation Multi-Armed Bandits

Knowing The What But Not The Where in Bayesian Optimization

1 code implementation ICML 2020 Vu Nguyen, Michael A. Osborne

In this paper, we consider a new setting in BO in which the knowledge of the optimum output f* is available.

Bayesian Optimization

Automated Model Selection with Bayesian Quadrature

no code implementations26 Feb 2019 Henry Chai, Jean-Francois Ton, Roman Garnett, Michael A. Osborne

We present a novel technique for tailoring Bayesian quadrature (BQ) to model selection.

Model Selection

ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems

2 code implementations17 Feb 2019 Philippe Wenk, Gabriele Abbati, Michael A. Osborne, Bernhard Schölkopf, Andreas Krause, Stefan Bauer

Parameter inference in ordinary differential equations is an important problem in many applied sciences and in engineering, especially in a data-scarce setting.

Gaussian Processes Model Selection +1

Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation

1 code implementation29 Jan 2019 Ahsan S. Alvi, Binxin Ru, Jan Calliess, Stephen J. Roberts, Michael A. Osborne

Batch Bayesian optimisation (BO) has been successfully applied to hyperparameter tuning using parallel computing, but it is wasteful of resources: workers that complete jobs ahead of others are left idle.

Bayesian Optimisation

Rejoinder for "Probabilistic Integration: A Role in Statistical Computation?"

no code implementations26 Nov 2018 Francois-Xavier Briol, Chris. J. Oates, Mark Girolami, Michael A. Osborne, Dino Sejdinovic

This article is the rejoinder for the paper "Probabilistic Integration: A Role in Statistical Computation?"

Battery health prediction under generalized conditions using a Gaussian process transition model

no code implementations17 Jul 2018 Robert R. Richardson, Michael A. Osborne, David A. Howey

Accurately predicting the future health of batteries is necessary to ensure reliable operation, minimise maintenance costs, and calculate the value of energy storage investments.

feature selection

Fingerprint Policy Optimisation for Robust Reinforcement Learning

no code implementations27 May 2018 Supratik Paul, Michael A. Osborne, Shimon Whiteson

Policy gradient methods ignore the potential value of adjusting environment variables: unobservable state features that are randomly determined by the environment in a physical setting, but are controllable in a simulator.

Bayesian Optimisation Continuous Control +3

Optimization, fast and slow: optimally switching between local and Bayesian optimization

1 code implementation ICML 2018 Mark McLeod, Michael A. Osborne, Stephen J. Roberts

We develop the first Bayesian Optimization algorithm, BLOSSOM, which selects between multiple alternative acquisition functions and traditional local optimization at each step.

Bayesian Optimization

Bayesian Optimization for Dynamic Problems

no code implementations9 Mar 2018 Favour M. Nyikosa, Michael A. Osborne, Stephen J. Roberts

We propose practical extensions to Bayesian optimization for solving dynamic problems.

Bayesian Optimization

Fast Information-theoretic Bayesian Optimisation

1 code implementation ICML 2018 Binxin Ru, Mark McLeod, Diego Granziol, Michael A. Osborne

Information-theoretic Bayesian optimisation techniques have demonstrated state-of-the-art performance in tackling important global optimisation problems.

Bayesian Optimisation

Bayesian Optimization for Probabilistic Programs

2 code implementations NeurIPS 2016 Tom Rainforth, Tuan Anh Le, Jan-Willem van de Meent, Michael A. Osborne, Frank Wood

We present the first general purpose framework for marginal maximum a posteriori estimation of probabilistic program variables.

Bayesian Optimization

Distributionally Ambiguous Optimization Techniques for Batch Bayesian Optimization

1 code implementation13 Jul 2017 Nikitas Rontsis, Michael A. Osborne, Paul J. Goulart

Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multivariate piecewise affine function.

Bayesian Optimization

A Novel Approach to Forecasting Financial Volatility with Gaussian Process Envelopes

no code implementations2 May 2017 Syed Ali Asad Rizvi, Stephen J. Roberts, Michael A. Osborne, Favour Nyikosa

In this paper we use Gaussian Process (GP) regression to propose a novel approach for predicting volatility of financial returns by forecasting the envelopes of the time series.

regression Time Series +1

Distribution of Gaussian Process Arc Lengths

no code implementations23 Mar 2017 Justin D. Bewsher, Alessandra Tosi, Michael A. Osborne, Stephen J. Roberts

We fill the gap in the existing literature by deriving the moments of the arc length for a stationary GP with multiple output dimensions.

Gaussian process regression for forecasting battery state of health

no code implementations16 Mar 2017 Robert R. Richardson, Michael A. Osborne, David A. Howey

Accurately predicting the future capacity and remaining useful life of batteries is necessary to ensure reliable system operation and to minimise maintenance costs.


Practical Bayesian Optimization for Variable Cost Objectives

1 code implementation13 Mar 2017 Mark McLeod, Michael A. Osborne, Stephen J. Roberts

We propose a novel Bayesian Optimization approach for black-box functions with an environmental variable whose value determines the tradeoff between evaluation cost and the fidelity of the evaluations.

Bayesian Optimization

Alternating Optimisation and Quadrature for Robust Control

no code implementations24 May 2016 Supratik Paul, Konstantinos Chatzilygeroudis, Kamil Ciosek, Jean-Baptiste Mouret, Michael A. Osborne, Shimon Whiteson

ALOQ is robust to the presence of significant rare events, which may not be observable under random sampling, but play a substantial role in determining the optimal policy.

Bayesian Optimisation

Preconditioning Kernel Matrices

1 code implementation22 Feb 2016 Kurt Cutajar, Michael A. Osborne, John P. Cunningham, Maurizio Filippone

Preconditioning is a common approach to alleviating this issue.

Probabilistic Integration: A Role in Statistical Computation?

no code implementations3 Dec 2015 François-Xavier Briol, Chris. J. Oates, Mark Girolami, Michael A. Osborne, Dino Sejdinovic

A research frontier has emerged in scientific computation, wherein numerical error is regarded as a source of epistemic uncertainty that can be modelled.

Numerical Integration

Blitzkriging: Kronecker-structured Stochastic Gaussian Processes

no code implementations27 Oct 2015 Thomas Nickson, Tom Gunter, Chris Lloyd, Michael A. Osborne, Stephen Roberts

We present Blitzkriging, a new approach to fast inference for Gaussian processes, applicable to regression, optimisation and classification.

Gaussian Processes General Classification +3

A Variational Bayesian State-Space Approach to Online Passive-Aggressive Regression

no code implementations8 Sep 2015 Arnold Salas, Stephen J. Roberts, Michael A. Osborne

Online Passive-Aggressive (PA) learning is a class of online margin-based algorithms suitable for a wide range of real-time prediction tasks, including classification and regression.

General Classification regression +1

Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees

no code implementations NeurIPS 2015 François-Xavier Briol, Chris. J. Oates, Mark Girolami, Michael A. Osborne

There is renewed interest in formulating integration as an inference problem, motivated by obtaining a full distribution over numerical error that can be propagated through subsequent computation.

Probabilistic Numerics and Uncertainty in Computations

no code implementations3 Jun 2015 Philipp Hennig, Michael A. Osborne, Mark Girolami

We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations.


Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature

no code implementations NeurIPS 2014 Tom Gunter, Michael A. Osborne, Roman Garnett, Philipp Hennig, Stephen J. Roberts

We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks.

Active Learning Numerical Integration

Variational Inference for Gaussian Process Modulated Poisson Processes

no code implementations2 Nov 2014 Chris Lloyd, Tom Gunter, Michael A. Osborne, Stephen J. Roberts

We present the first fully variational Bayesian inference scheme for continuous Gaussian-process-modulated Poisson processes.

Astronomy Bayesian Inference +2

Automated Machine Learning on Big Data using Stochastic Algorithm Tuning

no code implementations30 Jul 2014 Thomas Nickson, Michael A. Osborne, Steven Reece, Stephen J. Roberts

However, the state of the art in Bayesian optimisation is incapable of scaling to the large number of evaluations of algorithm performance required to fit realistic models to complex, big data.

Bayesian Optimisation Benchmarking +3

Efficient Bayesian Nonparametric Modelling of Structured Point Processes

no code implementations25 Jul 2014 Tom Gunter, Chris Lloyd, Michael A. Osborne, Stephen J. Roberts

This paper presents a Bayesian generative model for dependent Cox point processes, alongside an efficient inference scheme which scales as if the point processes were modelled independently.

Point Processes

Active Learning of Linear Embeddings for Gaussian Processes

no code implementations24 Oct 2013 Roman Garnett, Michael A. Osborne, Philipp Hennig

We propose an active learning method for discovering low-dimensional structure in high-dimensional Gaussian process (GP) tasks.

Active Learning Bayesian Optimization +2

A Kernel for Hierarchical Parameter Spaces

no code implementations21 Oct 2013 Frank Hutter, Michael A. Osborne

We define a family of kernels for mixed continuous/discrete hierarchical parameter spaces and show that they are positive definite.

Cannot find the paper you are looking for? You can Submit a new open access paper.