Search Results for author:
Found 109 papers, 39 papers with code
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
We introduce RecurrentGemma, an open language model which uses Google's novel Griffin architecture.
Gemma: Open Models Based on Gemini Research and Technology
This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models.
Plex: Towards Reliability using Pretrained Large Model Extensions
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures.
Pre-training helps Bayesian optimization too
Contrary to a common belief that BO is suited to optimizing black-box functions, it actually requires domain knowledge on characteristics of those functions to deploy BO successfully.
Neural Diffusion Processes
Neural network approaches for meta-learning distributions over functions have desirable properties such as increased flexibility and a reduced complexity of inference.
Pre-trained Gaussian processes for Bayesian optimization
Contrary to a common expectation that BO is suited to optimizing black-box functions, it actually requires domain knowledge about those functions to deploy BO successfully.
Deep Neural Networks as Point Estimates for Deep Gaussian Processes
This results in models that can either be seen as neural networks with improved uncertainty prediction or deep Gaussian processes with increased prediction accuracy.
Metropolis Algorithms for Representative Subgraph Sampling
While data mining in chemoinformatics studied graph data with dozens of nodes, systems biology and the Internet are now generating graph data with thousands and millions of nodes.
Learning Continuous Treatment Policy and Bipartite Embeddings for Matching with Heterogeneous Causal Effects
We propose to formulate the effectiveness of treatment as a parametrizable model, expanding to a multitude of treatment intensities and complexities through the continuous policy treatment function, and the likelihood of matching.
Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits
Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines.
DynamicPPL: Stan-like Speed for Dynamic Probabilistic Models
Since DynamicPPL is a modular, stand-alone library, any probabilistic programming system written in Julia, such as Turing. jl, can use DynamicPPL to specify models and trace their model parameters.
Resource-Efficient Neural Networks for Embedded Systems
While machine learning is traditionally a resource intensive task, embedded systems, autonomous navigation, and the vision of the Internet of Things fuel the interest in resource-efficient approaches.
AdvancedHMC.jl: A robust, modular and efficient implementation of advanced HMC algorithms
Stan's Hamilton Monte Carlo (HMC) has demonstrated remarkable sampling robustness and efficiency in a wide range of Bayesian inference problems through carefully crafted adaption schemes to the celebrated No-U-Turn sampler (NUTS) algorithm.
Bayesian Learning of Sum-Product Networks
While parameter learning in SPNs is well developed, structure learning leaves something to be desired: Even though there is a plethora of SPN structure learners, most of them are somewhat ad-hoc and based on intuition rather than a clear learning principle.
Efficient and Robust Machine Learning for Real-World Systems
In that way, we provide an extensive overview of the current state-of-the-art of robust and efficient machine learning for real-world systems.
MetaGAN: An Adversarial Approach to Few-Shot Learning
In this paper, we propose a conceptually simple and general framework called MetaGAN for few-shot learning problems.
Probabilistic Meta-Representations Of Neural Networks
Existing Bayesian treatments of neural networks are typically characterized by weak prior and approximate posterior distributions according to which all the weights are drawn independently.
Ergodic Measure Preserving Flows
Training probabilistic models with neural network components is intractable in most cases and requires to use approximations such as Markov chain Monte Carlo (MCMC), which is not scalable and requires significant hyper-parameter tuning, or mean-field variational inference (VI), which is biased.
Automatic Bayesian Density Analysis
Classical approaches for {exploratory data analysis} are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models and homogeneous likelihoods; they are sensitive to missing, corrupt and anomalous data; moreover, their expressiveness generally comes at the price of intractable inference.
Handling Incomplete Heterogeneous Data using VAEs
Variational autoencoders (VAEs), as well as other generative models, have been shown to be efficient and accurate for capturing the latent structure of vast amounts of complex high-dimensional data.
Variational Bayesian dropout: pitfalls and fixes
Dropout, a stochastic regularisation technique for training of neural networks, has recently been reinterpreted as a specific type of approximate inference algorithm for Bayesian neural networks.
Antithetic and Monte Carlo kernel estimators for partial rankings
We also present a novel variance reduction scheme based on an antithetic variate construction between permutations to obtain an improved estimator for the Mallows kernel.
Discovering Interpretable Representations for Both Deep Generative and Discriminative Models
We use a generative model which takes as input the representation in an existing (generative or discriminative) model, weakly supervised by limited side information.
Probabilistic Deep Learning using Random Sum-Product Networks
The need for consistent treatment of uncertainty has recently triggered increased interest in probabilistic deep learning methods.
Gaussian Process Behaviour in Wide Deep Neural Networks
Whilst deep neural networks have shown great empirical success, there is still much work to be done to understand their theoretical properties.
The Mirage of Action-Dependent Baselines in Reinforcement Learning
Policy gradient methods are a widely used class of model-free reinforcement learning algorithms where a state-dependent baseline is used to reduce gradient estimator variance.
Weakly supervised collective feature learning from curated media
Through these insights, we can define human curated groups as weak labels from which our proposed framework can learn discriminative features as a representation in the space of semantic concepts the users intended when creating the groups.
Few-shot learning of neural networks from scratch by pseudo example optimization
In this paper, we propose a simple but effective method for training neural networks with a limited amount of training data.
Variational Gaussian Dropout is not Bayesian
Gaussian multiplicative noise is commonly used as a stochastic regularisation technique in training of deterministic neural networks.
Automatic Discovery of the Statistical Types of Variables in a Dataset
A common practice in statistics and machine learning is to assume that the statistical data types (e. g., ordinal, categorical or real-valued) of variables, and usually also the likelihood model, is known.
A Birth-Death Process for Feature Allocation
We propose a Bayesian nonparametric prior over feature allocations for sequential data, the birth-death feature allocation process (BDFP).
General Latent Feature Modeling for Data Exploration Tasks
This paper introduces a general Bayesian non- parametric latent feature model suitable to per- form automatic exploratory analysis of heterogeneous datasets, where the attributes describing each object can be either discrete, continuous or mixed variables.
Improving Output Uncertainty Estimation and Generalization in Deep Learning via Neural Network Gaussian Processes
We propose a simple method that combines neural networks and Gaussian processes.
One-Shot Learning in Discriminative Neural Networks
We consider the task of one-shot learning of visual categories.
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
However, they often do not capture their own uncertainties well making them less robust in the real world as they overconfidently extrapolate and do not notice domain shift.
Lost Relatives of the Gumbel Trick
We show how a subfamily of our new methods adapts to this setting, proving new upper and lower bounds on the log partition function and deriving a family of sequential samplers for the Gibbs distribution.
General Latent Feature Models for Heterogeneous Datasets
Second, its Bayesian nonparametric nature allows us to automatically infer the model complexity from the data, i. e., the number of features necessary to capture the latent structure in the data.
Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning
Off-policy model-free deep reinforcement learning methods using previously collected data can improve sample efficiency over on-policy policy gradient techniques.
Deep Bayesian Active Learning with Image Data
In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way.
Bayesian inference on random simple graphs with power law degree distributions
The BFRY random variables are well approximated by gamma random variables in a variational Bayesian inference routine, which we apply to several network datasets for which power law degree distributions are a natural assumption.
Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
We analyze the connection between Q-Prop and existing model-free algorithms, and use control variate theory to derive two variants of Q-Prop with conservative and aggressive adaptation.
GPflow: A Gaussian process library using TensorFlow
GPflow is a Gaussian process library that uses TensorFlow for its core computations and Python for its front end.
A study of the effect of JPG compression on adversarial images
For Fast-Gradient-Sign perturbations of small magnitude, we found that JPG compression often reverses the drop in classification accuracy to a large extent, but not always.
Magnetic Hamiltonian Monte Carlo
We establish a theoretical basis for the use of non-canonical Hamiltonian dynamics in MCMC, and construct a symplectic, leapfrog-like integrator allowing for the implementation of magnetic HMC.
The Mondrian Kernel
We introduce the Mondrian kernel, a fast random feature approximation to the Laplace kernel.
Distributed Flexible Nonlinear Tensor Factorization
Tensor factorization is a powerful tool to analyse multi-way data.
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
Recent results at the intersection of Bayesian modelling and deep learning offer a Bayesian interpretation of common deep learning techniques such as dropout.
Ranked #35 on
Language Modelling
on Penn Treebank (Word Level)
Particle Gibbs for Infinite Hidden Markov Models
Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system.
Statistical Model Criticism using Kernel Two Sample Tests
We propose an exploratory approach to statistical model criticism using maximum mean discrepancy (MMD) two sample tests.
A General Framework for Constrained Bayesian Optimization using Information-based Search
Of particular interest to us is to efficiently solve problems with decoupled constraints, in which subsets of the objective and constraint functions may be evaluated independently.
Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
We develop parallel predictive entropy search (PPES), a novel algorithm for Bayesian optimization of expensive black-box objective functions.
Sandwiching the marginal likelihood using bidirectional Monte Carlo
Using the ground truth log-ML estimates obtained from our method, we quantitatively evaluate a wide variety of existing ML estimators on several latent variable models: clustering, a low rank approximation, and a binary attributes model.
Dirichlet Fragmentation Processes
In this paper, first we review the theory of random fragmentation processes [Bertoin, 2006], and a number of existing methods for modelling trees, including the popular nested Chinese restaurant process (nCRP).
Scalable Discrete Sampling as a Multi-Armed Bandit Problem
Drawing a sample from a discrete distribution is one of the building components for Monte Carlo methods.
An Empirical Study of Stochastic Variational Algorithms for the Beta Bernoulli Process
Stochastic variational inference (SVI) is emerging as the most promising candidate for scaling inference in Bayesian probabilistic models to large datasets.
MCMC for Variationally Sparse Gaussian Processes
This paper simultaneously addresses these, using a variational approximation to the posterior which is sparse in support of the function but otherwise free-form.
Neural Adaptive Sequential Monte Carlo
Experiments indicate that NASMC significantly improves inference in a non-linear state space model outperforming adaptive proposal methods including the Extended Kalman and Unscented Particle Filters.
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computational cost.
Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
Convolutional neural networks (CNNs) work well on large datasets.
Dropout as a Bayesian Approximation: Appendix
We show that a neural network with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to a well known Bayesian model.
Training generative neural networks via Maximum Mean Discrepancy optimization
We frame learning as an optimization minimizing a two-sample test statistic---informally speaking, a good generator network produces samples that cause a two-sample test to fail to reject the null hypothesis.
A Linear-Time Particle Gibbs Sampler for Infinite Hidden Markov Models
Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system.
On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes
We then discuss augmented index sets and show that, contrary to previous works, marginal consistency of augmentation is not enough to guarantee consistency of variational inference with the original model.
Latent Gaussian Processes for Distribution Estimation of Multivariate Categorical Data
Building on these ideas we propose a Bayesian model for the unsupervised task of distribution estimation of multivariate categorical data.
Predictive Entropy Search for Bayesian Optimization with Unknown Constraints
Unknown constraints arise in many types of expensive black-box optimization problems.
Slice Sampling for Probabilistic Programming
We introduce the first, general purpose, slice sampling inference engine for probabilistic programs.
General Table Completion using a Bayesian Nonparametric Model
Even though heterogeneous databases can be found in a broad variety of applications, there exists a lack of tools for estimating missing data in such databases.
Scalable Variational Gaussian Process Classification
Gaussian process classification is a popular method with a number of appealing properties.
Sublinear-Time Approximate MCMC Transitions for Probabilistic Programs
Probabilistic programming languages can simplify the development of machine learning techniques, but only if inference is sufficiently scalable.
Beta diffusion trees and hierarchical feature allocations
We define the beta diffusion tree, a random tree structure with a set of leaves that defines a collection of overlapping subsets of objects, known as a feature allocation.
Warped Mixtures for Nonparametric Cluster Shapes
A mixture of Gaussians fit to a single curved or heavy-tailed cluster will report that the data contains many clusters.
Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES).
Linear Dimensionality Reduction: Survey, Insights, and Generalizations
Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal low-dimensional projection of the data.
Classification using log Gaussian Cox processes
McCullagh and Yang (2006) suggest a family of classification algorithms based on Cox processes.
A reversible infinite HMM using normalised random measures
We present a nonparametric prior over reversible Markov chains.
Avoiding pathologies in very deep networks
Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance.
Student-t Processes as Alternatives to Gaussian Processes
We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions.
The Random Forest Kernel and other kernels for big data from random partitions
We present Random Partition Kernels, a new class of kernels derived by demonstrating a natural connection between random partitions of objects and kernels between those objects.
Automatic Construction and Natural-Language Description of Nonparametric Regression Models
This paper presents the beginnings of an automatic statistician, focusing on regression problems.
Gaussian Process Volatility Model
A Gaussian Process (GP) defines a distribution over functions, which allows us to capture highly flexible functional relationships for the variances.
Randomized Nonlinear Component Analysis
Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale.
The Supervised IBP: Neighbourhood Preserving Infinite Latent Feature Models
We propose a probabilistic model to infer supervised latent variables in the Hamming space from observed data.
Determinantal Clustering Processes - A Nonparametric Bayesian Approach to Kernel Based Semi-Supervised Clustering
Semi-supervised clustering is the task of clustering data points into clusters where only a fraction of the points are labelled.
Bayesian Structured Prediction Using Gaussian Processes
We introduce a conceptually novel structured prediction model, GPstruct, which is kernelized, non-parametric and Bayesian, by design.
Dynamic Covariance Models for Multivariate Financial Time Series
The accurate prediction of time-changing covariances is an important problem in the modeling of multivariate financial data.
Identifying cancer subtypes in glioblastoma by combining genomic, transcriptomic and epigenomic data
We apply the method to 277 glioblastoma samples from The Cancer Genome Atlas, for which there are gene expression, copy number variation, methylation and microRNA data.
Scaling the Indian Buffet Process via Submodular Maximization
Inference for latent feature models is inherently difficult as the inference space grows exponentially with the size of the input data and number of latent features.
A dependent partition-valued process for multitask clustering and time evolving network modelling
The fundamental aim of clustering algorithms is to partition data points.
Structure Discovery in Nonparametric Regression through Compositional Kernel Search
Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art.
Collaborative Gaussian Processes for Preference Learning
We present a new model based on Gaussian processes (GPs) for learning pairwise preferences expressed by multiple users.
A nonparametric variable clustering model
Factor analysis models effectively summarise the covariance structure of high dimensional data, but the solutions are typically hard to interpret.
Continuous Relaxations for Discrete Hamiltonian Monte Carlo
Continuous relaxations play an important role in discrete optimization, but have not seen much use in approximate probabilistic inference.
Active Learning of Model Evidence Using Bayesian Quadrature
Numerical integration is an key component of many problems in scientific computing, statistical modelling, and machine learning.
SiGMa: Simple Greedy Matching for Aligning Large Knowledge Bases
The Internet has enabled the creation of a growing number of large-scale knowledge bases in a variety of domains containing complementary information.
Variable noise and dimensionality reduction for sparse Gaussian processes
A projection of the input space to a low dimensional space is learned in a supervised manner, alongside the pseudo-inputs, which now live in this reduced space.
Warped Mixtures for Nonparametric Cluster Shapes
A mixture of Gaussians fit to a single curved or heavy-tailed cluster will report that the data contains many clusters.
Bayesian Active Learning for Classification and Preference Learning
Information theoretic active learning has been widely studied for probabilistic models.
Testing a Bayesian Measure of Representativeness Using a Large Image Database
How do people determine which elements of a set are most representative of that set?
Gaussian Process Regression Networks
We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes.
Generalised Wishart Processes
We introduce a stochastic process with Wishart marginals: the generalised Wishart process (GWP).
Tree-Structured Stick Breaking for Hierarchical Data
Many data are naturally modeled by an unobserved hierarchical structure.
Copula Processes
We define a copula process which describes the dependencies between arbitrarily many random variables independently of their marginal distributions.
Ranking relations using analogies in biological and information networks
Our work addresses the following question: is the relation between objects A and B analogous to those relations found in $\mathbf{S}$?
Large Scale Nonparametric Bayesian Inference: Data Parallelisation in the Indian Buffet Process
Nonparametric Bayesian models provide a framework for flexible probabilistic modelling of complex datasets.
The Infinite Factorial Hidden Markov Model
We introduces a new probability distribution over a potentially infinite number of binary Markov chains which we call the Markov Indian buffet process.
Bayesian Exponential Family PCA
Principal Components Analysis (PCA) has become established as one of the key tools for dimensionality reduction when dealing with real valued data.
Sparse Gaussian processes using pseudo-inputs
We present a new Gaussian process (GP) regression model whose covariance is parameterized by the the locations of M pseudo-input points, which we learn by a gradient based optimization.
