no code implementations • EMNLP (MRL) 2021 • Asa Cooper Stickland, Iain Murray
Many recent works use ‘consistency regularisation’ to improve the generalisation of fine-tuned pre-trained models, both multilingual and English-only.
1 code implementation • ICLR Workshop Neural_Compression 2021 • James Townsend, Iain Murray
We generalize the 'bits back with ANS' method to time-series models with a latent Markov structure.
3 code implementations • NeurIPS 2021 • Yang song, Conor Durkan, Iain Murray, Stefano Ermon
Score-based diffusion models synthesize samples by reversing a stochastic process that diffuses data to noise, and are trained by minimizing a weighted combination of score matching losses.
Ranked #6 on Image Generation on ImageNet 32x32 (bpd metric)
no code implementations • 8 Jul 2020 • Asa Cooper Stickland, Iain Murray
Modern deep neural networks can produce badly calibrated predictions, especially when train and test distributions are mismatched.
1 code implementation • 16 Jun 2020 • Tim Dockhorn, James A. Ritchie, Yao-Liang Yu, Iain Murray
Density deconvolution is the task of estimating a probability density function given only noise-corrupted samples.
1 code implementation • 15 Jun 2020 • Artur Bekasov, Iain Murray
Like in PCA, the leading latent dimensions define a sequence of manifolds that lie close to the data.
5 code implementations • ICML 2020 • Conor Durkan, Iain Murray, George Papamakarios
Likelihood-free methods perform parameter inference in stochastic simulator models where evaluating the likelihood is intractable but sampling synthetic data is possible.
1 code implementation • 26 Nov 2019 • James A. Ritchie, Iain Murray
The Extreme Deconvolution method fits a probability density to a dataset where each observation has Gaussian noise added with a known sample-specific covariance, originally intended for use with astronomical datasets.
no code implementations • 29 Jul 2019 • Chaoyun Zhang, Marco Fiore, Iain Murray, Paul Patras
This paper introduces CloudLSTM, a new branch of recurrent neural models tailored to forecasting over data streams generated by geospatial point-cloud sources.
8 code implementations • NeurIPS 2019 • Conor Durkan, Artur Bekasov, Iain Murray, George Papamakarios
A normalizing flow models a complex probability density as an invertible transformation of a simple base density.
no code implementations • 5 Jun 2019 • Conor Durkan, Artur Bekasov, Iain Murray, George Papamakarios
A normalizing flow models a complex probability density as an invertible transformation of a simple density.
1 code implementation • 17 Apr 2019 • Ben Krause, Emmanuel Kahembwe, Iain Murray, Steve Renals
This research note combines two methods that have recently improved the state of the art in language modeling: Transformers and dynamic evaluation.
Ranked #1 on Language Modelling on Hutter Prize
2 code implementations • 7 Feb 2019 • Asa Cooper Stickland, Iain Murray
Multi-task learning shares information between related tasks, sometimes reducing the number of parameters required.
no code implementations • 29 Nov 2018 • Artur Bekasov, Iain Murray
Modern deep neural network models suffer from adversarial examples, i. e. confidently misclassified points in the input space.
no code implementations • 21 Nov 2018 • Conor Durkan, George Papamakarios, Iain Murray
Likelihood-free inference refers to inference when a likelihood function cannot be explicitly evaluated, which is often the case for models based on simulators.
2 code implementations • ICLR 2019 • Lucas Deecke, Iain Murray, Hakan Bilen
Normalization methods are a central building block in the deep learning toolbox.
10 code implementations • 18 May 2018 • George Papamakarios, David C. Sterratt, Iain Murray
We present Sequential Neural Likelihood (SNL), a new method for Bayesian inference in simulator models, where the likelihood is intractable but simulating data from the model is possible.
1 code implementation • 13 Nov 2017 • Sohan Seth, Iain Murray, Christopher K. I. Williams
Model criticism is usually carried out by assessing if replicated data generated under the fitted model looks similar to the observed data, see e. g. Gelman, Carlin, Stern, and Rubin [2004, p. 165].
3 code implementations • ICML 2018 • Ben Krause, Emmanuel Kahembwe, Iain Murray, Steve Renals
We present methodology for using dynamic evaluation to improve neural sequence models.
Ranked #10 on Language Modelling on Hutter Prize
20 code implementations • NeurIPS 2017 • George Papamakarios, Theo Pavlakou, Iain Murray
By constructing a stack of autoregressive models, each modelling the random numbers of the next model in the stack, we obtain a type of normalizing flow suitable for density estimation, which we call Masked Autoregressive Flow.
Ranked #1 on Density Estimation on CIFAR-10 (Conditional)
no code implementations • EACL 2017 • Philippa Shoemark, Debnil Sur, Luke Shrimpton, Iain Murray, Sharon Goldwater
Political surveys have indicated a relationship between a sense of Scottish identity and voting decisions in the 2014 Scottish Independence Referendum.
1 code implementation • NeurIPS 2016 • George Papamakarios, Iain Murray
In some cases, learning an accurate parametric representation of the entire true posterior distribution requires fewer model simulations than Monte Carlo ABC methods need to produce a single sample from an approximate posterior.
no code implementations • 15 Oct 2016 • Colin Wei, Iain Murray
Computing partition functions, the normalizing constants of probability distributions, is often hard.
1 code implementation • 26 Sep 2016 • Ben Krause, Liang Lu, Iain Murray, Steve Renals
We introduce multiplicative LSTM (mLSTM), a recurrent neural network architecture for sequence modelling that combines the long short-term memory (LSTM) and multiplicative recurrent neural network architectures.
Ranked #14 on Language Modelling on Hutter Prize
1 code implementation • 20 May 2016 • George Papamakarios, Iain Murray
In some cases, learning an accurate parametric representation of the entire true posterior distribution requires fewer model simulations than Monte Carlo ABC methods need to produce a single sample from an approximate posterior.
3 code implementations • 7 May 2016 • Benigno Uria, Marc-Alexandre Côté, Karol Gregor, Iain Murray, Hugo Larochelle
We present Neural Autoregressive Distribution Estimation (NADE) models, which are neural network architectures applied to the problem of unsupervised distribution and density estimation.
4 code implementations • 24 Feb 2016 • Iain Murray
We review strategies for differentiating matrix-based computations, and derive symbolic and algorithmic update rules for differentiating expressions containing the Cholesky decomposition.
Computation Mathematical Software
17 code implementations • 12 Feb 2015 • Mathieu Germain, Karol Gregor, Iain Murray, Hugo Larochelle
There has been a lot of recent interest in designing neural network models to estimate a distribution from a set of examples.
Ranked #4 on Density Estimation on UCI GAS
no code implementations • 9 Aug 2014 • Ryan Prescott Adams, George E. Dahl, Iain Murray
Probabilistic matrix factorization (PMF) is a powerful method for modeling data associ- ated with pairwise relationships, Finding use in collaborative Filtering, computational bi- ology, and document analysis, among other areas.
no code implementations • 7 Oct 2013 • Benigno Uria, Iain Murray, Hugo Larochelle
We can thus use the most convenient model for each inference task at hand, and ensembles of such models with different orderings are immediately available.
Ranked #10 on Image Generation on Binarized MNIST
no code implementations • NeurIPS 2013 • Benigno Uria, Iain Murray, Hugo Larochelle
We introduce RNADE, a new model for joint density estimation of real-valued vectors.
no code implementations • 28 Oct 2012 • Robert Nishihara, Iain Murray, Ryan P. Adams
Probabilistic models are conceptually powerful tools for finding structure in data, but their practical effectiveness is often limited by our ability to perform inference in them.
no code implementations • NeurIPS 2011 • Jakob H. Macke, Iain Murray, Peter E. Latham
However, maximum entropy models fit to small data sets can be subject to sampling bias; i. e. the true entropy of the data can be severely underestimated.
no code implementations • NeurIPS 2010 • Iain Murray, Ryan P. Adams
The Gaussian process (GP) is a popular way to specify dependencies between random variables in a probabilistic model.
1 code implementation • 31 Dec 2009 • Iain Murray, Ryan Prescott Adams, David J. C. MacKay
Many probabilistic models introduce strong dependencies between variables using a latent multivariate Gaussian distribution or a Gaussian process.
no code implementations • NeurIPS 2008 • Iain Murray, Ruslan R. Salakhutdinov
We present a simple new Monte Carlo algorithm for evaluating probabilities of observations in complex latent variable models, such as Deep Belief Networks.
no code implementations • NeurIPS 2008 • Iain Murray, David Mackay, Ryan P. Adams
Samples drawn from the GPDS are consistent with exact, independent samples from a fixed density function that is a transformation of a function drawn from a Gaussian process prior.