Search Results for author:
Found 47 papers, 20 papers with code
On the Challenges and Opportunities in Generative AI
The field of deep generative modeling has grown rapidly and consistently over the years.
Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood
Neural network sparsification is a promising avenue to save computational time and memory costs, especially in an age where many successful AI models are becoming too large to na\"ively deploy on consumer hardware.
Position Paper: Bayesian Deep Learning in the Age of Large-Scale AI
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets.
Uncertainty in Graph Contrastive Learning with Bayesian Neural Networks
Graph contrastive learning has shown great promise when labeled data is scarce, but large unlabeled datasets are available.
Estimating optimal PAC-Bayes bounds with Hamiltonian Monte Carlo
An important yet underexplored question in the PAC-Bayes literature is how much tightness we lose by restricting the posterior family to factorized Gaussian distributions when optimizing a PAC-Bayes bound.
A Primer on Bayesian Neural Networks: Review and Debates
Neural networks have achieved remarkable performance across various problem domains, but their widespread applicability is hindered by inherent limitations such as overconfidence in predictions, lack of interpretability, and vulnerability to adversarial attacks.
Hodge-Aware Contrastive Learning
We leverage this decomposition to develop a contrastive self-supervised learning approach for processing simplicial data and generating embeddings that encapsulate specific spectral information. Specifically, we encode the pertinent data invariances through simplicial neural networks and devise augmentations that yield positive contrastive examples with suitable spectral properties for downstream tasks.
Understanding Pathologies of Deep Heteroskedastic Regression
Deep, overparameterized regression models are notorious for their tendency to overfit.
Improving Neural Additive Models with Bayesian Principles
Neural additive models (NAMs) enhance the transparency of deep neural networks by handling input features in separate additive sub-networks.
Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization
The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks.
Incorporating Unlabelled Data into Bayesian Neural Networks
Conventional Bayesian Neural Networks (BNNs) cannot leverage unlabelled data to improve their predictions.
Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practice
Meta-Learning aims to speed up the learning process on new tasks by acquiring useful inductive biases from datasets of related learning tasks.
Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations
We develop a convenient gradient-based method for selecting the data augmentation without validation data during training of a deep neural network.
Meta-learning richer priors for VAEs
Variational auto-encoders have proven to capture complicated data distributions and useful latent representations, while advances in meta-learning have made it possible to extract prior knowledge from data.
Probing as Quantifying Inductive Bias
Such performance improvements have motivated researchers to quantify and understand the linguistic information encoded in these representations.
Pathologies in priors and inference for Bayesian transformers
In recent years, the transformer has established itself as a workhorse in many applications ranging from natural language processing to reinforcement learning.
Sparse MoEs meet Efficient Ensembles
Machine learning models based on the aggregated outputs of submodels, either at the activation or prediction levels, often exhibit strong performance compared to individual models.
Deep Classifiers with Label Noise Modeling and Distance Awareness
Uncertainty estimation in deep learning has recently emerged as a crucial area of interest to advance reliability and robustness in safety-critical applications.
Neural Variational Gradient Descent
Particle-based approximate Bayesian inference approaches such as Stein Variational Gradient Descent (SVGD) combine the flexibility and convergence guarantees of sampling methods with the computational benefits of variational inference.
A Bayesian Approach to Invariant Deep Neural Networks
We propose a novel Bayesian neural network architecture that can learn invariances from data alone by inferring a posterior distribution over different weight-sharing schemes.
Repulsive Deep Ensembles are Bayesian
Deep ensembles have recently gained popularity in the deep learning community for their conceptual simplicity and efficiency.
On Stein Variational Neural Network Ensembles
Ensembles of deep neural networks have achieved great success recently, but they do not offer a proper Bayesian justification.
Data augmentation in Bayesian neural networks and the cold posterior effect
Here, we provide several approaches to developing principled Bayesian neural networks incorporating data augmentation.
Priors in Bayesian Deep Learning: A Review
While the choice of prior is one of the most critical parts of the Bayesian inference workflow, recent Bayesian deep learning models have often fallen back on vague priors, such as standard Gaussians.
BNNpriors: A library for Bayesian neural network inference with different prior distributions
Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance.
Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning
Marginal-likelihood based model-selection, even though promising, is rarely used in deep learning due to estimation difficulties.
Bayesian Neural Network Priors Revisited
Isotropic Gaussian priors are the de facto standard for modern Bayesian neural network inference.
On Disentanglement in Gaussian Process Variational Autoencoders
While many models have been introduced to learn such disentangled representations, only few attempt to explicitly exploit the structure of sequential data.
Exact Langevin Dynamics with Stochastic Gradients
Stochastic gradient Markov Chain Monte Carlo algorithms are popular samplers for approximate inference, but they are generally biased.
Annealed Stein Variational Gradient Descent
Particle based optimization algorithms have recently been developed as sampling methods that iteratively update a set of particles to approximate a target distribution.
Factorized Gaussian Process Variational Autoencoders
Variational autoencoders often assume isotropic Gaussian priors and mean-field posteriors, hence do not exploit structure in scenarios where we may expect similarity or consistency across latent variables.
Scalable Gaussian Process Variational Autoencoders
Conventional variational autoencoders fail in modeling correlations between data points due to their use of factorized priors.
Sparse Gaussian Process Variational Autoencoders
Large, multi-dimensional spatio-temporal datasets are omnipresent in modern science and engineering.
Conservative Uncertainty Estimation By Fitting Prior Networks
Obtaining high-quality uncertainty estimates is essential for many applications of deep neural networks.
PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees
Meta-learning can successfully acquire useful inductive biases from data.
Mixture-of-Experts Variational Autoencoder for Clustering and Generating from Similarity-Based Representations on Single Cell Data
Clustering high-dimensional data, such as images or biological measurements, is a long-standingproblem and has been studied extensively.
Deep Amortized Variational Inference for Multivariate Time Series Imputation with Latent Gaussian Process Models
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years.
Dimensionality Reduction
Multivariate Time Series Imputation
+3
DPSOM: Deep Probabilistic Clustering with Self-Organizing Maps
We show that DPSOM achieves superior clustering performance compared to current deep clustering methods on MNIST/Fashion-MNIST, while maintaining the favourable visualization properties of SOMs.
META$^\mathbf{2}$: Memory-efficient taxonomic classification and abundance estimation for metagenomics with deep learning
We therefore aim to develop a more memory-efficient technique for taxonomic classification.
MGP-AttTCN: An Interpretable Machine Learning Model for the Prediction of Sepsis
With a mortality rate of 5. 4 million lives worldwide every year and a healthcare cost of more than 16 billion dollars in the USA alone, sepsis is one of the leading causes of hospital mortality and an increasing concern in the ageing western world.
Deep Multiple Instance Learning for Taxonomic Classification of Metagenomic read sets
Of particular interest is the determination of the distribution of the taxa of microbes in metagenomic samples.
Variational pSOM: Deep Probabilistic Clustering with Self-Organizing Maps
There are very performant deep clustering models on the one hand and interpretable representation learning techniques, often relying on latent topological structures such as self-organizing maps, on the other hand.
Mixture-of-Experts Variational Autoencoder for clustering and generating from similarity-based representations
MoE-Sim-VAE is based on a Variational Autoencoder (VAE), where the decoder consists of a Mixture-of-Experts (MoE) architecture.
GP-VAE: Deep Probabilistic Time Series Imputation
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years.
Dimensionality Reduction
Multivariate Time Series Imputation
+2
Meta-Learning Mean Functions for Gaussian Processes
When it comes to meta-learning in Gaussian process models, approaches in this setting have mostly focused on learning the kernel function of the prior, but not on learning its mean function.
Scalable Gaussian Processes on Discrete Domains
We explore different techniques for selecting inducing points on discrete domains, including greedy selection, determinantal point processes, and simulated annealing.
SOM-VAE: Interpretable Discrete Representation Learning on Time Series
We evaluate our model in terms of clustering performance and interpretability on static (Fashion-)MNIST data, a time series of linearly interpolated (Fashion-)MNIST images, a chaotic Lorenz attractor system with two macro states, as well as on a challenging real world medical time series application on the eICU data set.
