Search Results for author: David Barber

Found 53 papers, 14 papers with code

Latent Attention for Linear Time Transformers

no code implementations27 Feb 2024 Rares Dolga, Marius Cobzarenco, David Barber

The time complexity of the standard attention mechanism in a transformer scales quadratically with the length of the sequence.

Text Generation

Mafin: Enhancing Black-Box Embeddings with Model Augmented Fine-Tuning

no code implementations19 Feb 2024 Mingtian Zhang, Shawn Lan, Peter Hayes, David Barber

Our results demonstrate that Mafin significantly enhances the performance of the black-box embeddings by only requiring the training of a small augmented model.

Retrieval

Active Preference Learning for Large Language Models

no code implementations12 Feb 2024 William Muldrew, Peter Hayes, Mingtian Zhang, David Barber

A key consideration for aligning these models is how to most effectively use human resources, or model resources in the case where LLMs themselves are used as oracles.

Active Learning Language Modelling

Diffusive Gibbs Sampling

1 code implementation5 Feb 2024 Wenlin Chen, Mingtian Zhang, Brooks Paige, José Miguel Hernández-Lobato, David Barber

The inadequate mixing of conventional Markov Chain Monte Carlo (MCMC) methods for multi-modal distributions presents a significant challenge in practical applications such as Bayesian inference and molecular dynamics.

Bayesian Inference

CenTime: Event-Conditional Modelling of Censoring in Survival Analysis

2 code implementations7 Sep 2023 Ahmed H. Shahin, An Zhao, Alexander C. Whitehead, Daniel C. Alexander, Joseph Jacob, David Barber

We demonstrate that our approach forms a consistent estimator for the event model parameters, even in the absence of uncensored data.

Survival Analysis

Moment Matching Denoising Gibbs Sampling

1 code implementation NeurIPS 2023 Mingtian Zhang, Alex Hawkins-Hooker, Brooks Paige, David Barber

Energy-Based Models (EBMs) offer a versatile framework for modeling complex data distributions.

Denoising

Generalized Multiple Intent Conditioned Slot Filling

no code implementations18 May 2023 Harshil Shah, Arthur Wilcke, Marius Cobzarenco, Cristi Cobzarenco, Edward Challis, David Barber

Natural language understanding includes the tasks of intent detection (identifying a user's objectives) and slot filling (extracting the entities relevant to those objectives).

Intent Detection Language Modelling +4

Smoothed Q-learning

no code implementations15 Mar 2023 David Barber

In Reinforcement Learning the Q-learning algorithm provably converges to the optimal solution.

Q-Learning reinforcement-learning +1

Towards Healing the Blindness of Score Matching

no code implementations15 Sep 2022 Mingtian Zhang, Oscar Key, Peter Hayes, David Barber, Brooks Paige, François-Xavier Briol

Score-based divergences have been widely used in machine learning and statistics applications.

Density Estimation

Integrated Weak Learning

no code implementations19 Jun 2022 Peter Hayes, Mingtian Zhang, Raza Habib, Jordan Burgess, Emine Yilmaz, David Barber

We introduce a label model that can learn to aggregate weak supervision sources differently for different datapoints and takes into consideration the performance of the end-model during training.

Improving VAE-based Representation Learning

no code implementations28 May 2022 Mingtian Zhang, Tim Z. Xiao, Brooks Paige, David Barber

Latent variable models like the Variational Auto-Encoder (VAE) are commonly used to learn representations of images.

Representation Learning

Generalization Gap in Amortized Inference

1 code implementation23 May 2022 Mingtian Zhang, Peter Hayes, David Barber

The ability of likelihood-based probabilistic models to generalize to unseen data is central to many machine learning applications such as lossless compression.

Survival Analysis for Idiopathic Pulmonary Fibrosis using CT Images and Incomplete Clinical Data

2 code implementations21 Mar 2022 Ahmed H. Shahin, Joseph Jacob, Daniel C. Alexander, David Barber

To this end, we propose a probabilistic model that captures the dependencies between the observed clinical variables and imputes missing ones.

Imputation Survival Analysis

Parallel Neural Local Lossless Compression

2 code implementations13 Jan 2022 Mingtian Zhang, James Townsend, Ning Kang, David Barber

The recently proposed Neural Local Lossless Compression (NeLLoC), which is based on a local autoregressive model, has achieved state-of-the-art (SOTA) out-of-distribution (OOD) generalization performance in the image compression task.

Image Compression

Adaptive Optimization with Examplewise Gradients

1 code implementation30 Nov 2021 Julius Kunze, James Townsend, David Barber

We propose a new, more general approach to the design of stochastic gradient-based optimization methods for machine learning.

BIG-bench Machine Learning

Sample Efficient Model Evaluation

no code implementations24 Sep 2021 Emine Yilmaz, Peter Hayes, Raza Habib, Jordan Burgess, David Barber

Labelling data is a major practical bottleneck in training and testing classifiers.

Locally-Contextual Nonlinear CRFs for Sequence Labeling

no code implementations30 Mar 2021 Harshil Shah, Tim Xiao, David Barber

Linear chain conditional random fields (CRFs) combined with contextual word embeddings have achieved state of the art performance on sequence labeling tasks.

Chunking named-entity-recognition +3

Efficiently labelling sequences using semi-supervised active learning

no code implementations1 Jan 2021 Harshil Shah, David Barber

However, active learning methods usually use supervised training and ignore the data points which have not yet been labelled.

Active Learning Missing Labels

{Learning disentangled representations with the Wasserstein Autoencoder

no code implementations1 Jan 2021 Benoit Gaujac, Ilya Feige, David Barber

We further study the trade off between disentanglement and reconstruction on more-difficult data sets with unknown generative factors, where we expect improved reconstructions due to the flexibility of the WAE paradigm.

Disentanglement

Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks

no code implementations ICLR 2021 Thomas Bird, Friso H. Kingma, David Barber

In this work we show, for the first time, that we can successfully train generative models which utilize binary neural networks.

Representation Learning for High-Dimensional Data Collection under Local Differential Privacy

no code implementations23 Oct 2020 Alex Mansbridge, Gregory Barbour, Davide Piras, Michael Murray, Christopher Frye, Ilya Feige, David Barber

In this work, our contributions are two-fold: first, by adapting state-of-the-art techniques from representation learning, we introduce a novel approach to learning LDP mechanisms.

Denoising Representation Learning +1

Learning disentangled representations with the Wasserstein Autoencoder

no code implementations7 Oct 2020 Benoit Gaujac, Ilya Feige, David Barber

We further study the trade off between disentanglement and reconstruction on more-difficult data sets with unknown generative factors, where the flexibility of the WAE paradigm in the reconstruction term improves reconstructions.

Disentanglement

Learning Deep-Latent Hierarchies by Stacking Wasserstein Autoencoders

no code implementations7 Oct 2020 Benoit Gaujac, Ilya Feige, David Barber

Probabilistic models with hierarchical-latent-variable structures provide state-of-the-art results amongst non-autoregressive, unsupervised density-based models.

Bayesian Online Meta-Learning

no code implementations28 Sep 2020 Pauching Yap, Hippolyt Ritter, David Barber

This work introduces a Bayesian online meta-learning framework to tackle the catastrophic forgetting and the sequential few-shot tasks problems.

Classification Meta-Learning +1

Addressing Catastrophic Forgetting in Few-Shot Problems

1 code implementation30 Apr 2020 Pauching Yap, Hippolyt Ritter, David Barber

We demonstrate that the popular gradient-based model-agnostic meta-learning algorithm (MAML) indeed suffers from catastrophic forgetting and introduce a Bayesian online meta-learning framework that tackles this problem.

Classification General Classification +2

Private Machine Learning via Randomised Response

no code implementations14 Jan 2020 David Barber

We introduce a general learning framework for private machine learning based on randomised response.

BIG-bench Machine Learning regression

HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models

1 code implementation ICLR 2020 James Townsend, Thomas Bird, Julius Kunze, David Barber

We make the following striking observation: fully convolutional VAE models trained on 32x32 ImageNet can generalize well, not just to 64x64 but also to far larger photographs, with no changes to the model.

Image Compression

Variational f-divergence Minimization

no code implementations27 Jul 2019 Mingtian Zhang, Thomas Bird, Raza Habib, Tianlin Xu, David Barber

Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific f-divergence between the model and data distribution.

Image Generation

Auxiliary Variational MCMC

1 code implementation ICLR 2019 Raza Habib, David Barber

We introduce Auxiliary Variational MCMC, a novel framework for learning MCMC kernels that combines recent advances in variational inference with insights drawn from traditional auxiliary variable MCMC methods such as Hamiltonian Monte Carlo.

regression Variational Inference

Gaussian Mean Field Regularizes by Limiting Learned Information

no code implementations12 Feb 2019 Julius Kunze, Louis Kirsch, Hippolyt Ritter, David Barber

Variational inference with a factorized Gaussian posterior estimate is a widely used approach for learning parameters and hidden variables.

Variational Inference

Spread Divergence

no code implementations21 Nov 2018 Mingtian Zhang, Peter Hayes, Tom Bird, Raza Habib, David Barber

For distributions $\mathbb{P}$ and $\mathbb{Q}$ with different supports or undefined densities, the divergence $\textrm{D}(\mathbb{P}||\mathbb{Q})$ may not exist.

Noisy Information Bottlenecks for Generalization

no code implementations27 Sep 2018 Julius Kunze, Louis Kirsch, Hippolyt Ritter, David Barber

We propose Noisy Information Bottlenecks (NIB) to limit mutual information between learned parameters and the data through noise.

Training generative latent models by variational f-divergence minimization

no code implementations27 Sep 2018 Mingtian Zhang, Thomas Bird, Raza Habib, Tianlin Xu, David Barber

Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific form of f-divergence between the model and data distribution.

Stochastic Variational Optimization

no code implementations13 Sep 2018 Thomas Bird, Julius Kunze, David Barber

These approaches are of particular interest because they are parallelizable.

Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers

1 code implementation CVPR 2019 Zhen He, Jian Li, Daxue Liu, Hangen He, David Barber

To achieve both label-free and end-to-end learning of MOT, we propose a Tracking-by-Animation framework, where a differentiable neural model first tracks objects from input frames and then animates these objects into reconstructed frames.

Multi-Object Tracking Online Multi-Object Tracking

Generative Neural Machine Translation

no code implementations NeurIPS 2018 Harshil Shah, David Barber

We introduce Generative Neural Machine Translation (GNMT), a latent variable architecture which is designed to model the semantics of the source and target sentences.

Machine Translation Sentence +1

Generating Sentences Using a Dynamic Canvas

no code implementations13 Jun 2018 Harshil Shah, Bowen Zheng, David Barber

We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word level generative model for natural language.

Sentence

Improving latent variable descriptiveness with AutoGen

no code implementations12 Jun 2018 Alex Mansbridge, Roberto Fierimonte, Ilya Feige, David Barber

Powerful generative models, particularly in Natural Language Modelling, are commonly trained by maximizing a variational lower bound on the data log likelihood.

Language Modelling

Gaussian mixture models with Wasserstein distance

no code implementations12 Jun 2018 Benoit Gaujac, Ilya Feige, David Barber

Generative models with both discrete and continuous latent variables are highly motivated by the structure of many real-world data sets.

Descriptive

Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting

no code implementations NeurIPS 2018 Hippolyt Ritter, Aleksandar Botev, David Barber

In order to make our method scalable, we leverage recent block-diagonal Kronecker factored approximations to the curvature.

Permuted-MNIST

A Scalable Laplace Approximation for Neural Networks

1 code implementation ICLR 2018 Hippolyt Ritter, Aleksandar Botev, David Barber

Pytorch implementations of Bayes By Backprop, MC Dropout, SGLD, the Local Reparametrization Trick, KF-Laplace and more

Bayesian Inference

Practical Gauss-Newton Optimisation for Deep Learning

no code implementations ICML 2017 Aleksandar Botev, Hippolyt Ritter, David Barber

We present an efficient block-diagonal ap- proximation to the Gauss-Newton matrix for feedforward neural networks.

Thinking Fast and Slow with Deep Learning and Tree Search

4 code implementations NeurIPS 2017 Thomas Anthony, Zheng Tian, David Barber

Sequential decision making problems, such as structured prediction, robotic control, and game playing, require a combination of planning policies and generalisation of those plans.

Decision Making reinforcement-learning +2

Nesterov's Accelerated Gradient and Momentum as approximations to Regularised Update Descent

no code implementations7 Jul 2016 Aleksandar Botev, Guy Lever, David Barber

We present a unifying framework for adapting the update direction in gradient-based iterative optimization methods.

Dealing with a large number of classes -- Likelihood, Discrimination or Ranking?

no code implementations22 Jun 2016 David Barber, Aleksandar Botev

We consider training probabilistic classifiers in the case of a large number of classes.

On solving Ordinary Differential Equations using Gaussian Processes

no code implementations17 Aug 2014 David Barber

We describe a set of Gaussian Process based approaches that can be used to solve non-linear Ordinary Differential Equations.

Gaussian Processes

Affine Independent Variational Inference

no code implementations NeurIPS 2012 Edward Challis, David Barber

We present a method for approximate inference for a broad class of non-conjugate probabilistic models.

Variational Inference

A Unifying Perspective of Parametric Policy Search Methods for Markov Decision Processes

no code implementations NeurIPS 2012 Thomas Furmston, David Barber

This analysis leads naturally to the consideration of this approximate Newton method as an alternative gradient-based method for Markov Decision Processes.

Cannot find the paper you are looking for? You can Submit a new open access paper.