Search Results for author:
Found 53 papers, 26 papers with code
Mutual Information Multinomial Estimation
Estimating mutual information (MI) is a fundamental yet challenging task in data science and machine learning.
Identifying Nonstationary Causal Structures with High-Order Markov Switching Models
Causal discovery in time series is a rapidly evolving field with a wide variety of applications in other areas such as climate science and neuroscience.
Diffusion Model With Optimal Covariance Matching
The probabilistic diffusion model has become highly effective across various domains.
C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion
Through a series of observations, we find that the prompt choice significantly affects the calibration in CLIP, where the prompts leading to higher text feature dispersion result in better-calibrated predictions.
On the Challenges and Opportunities in Generative AI
The field of deep generative modeling has grown rapidly and consistently over the years.
Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets.
Training Discrete Energy-Based Models with Energy Discrepancy
Training energy-based models (EBMs) on discrete spaces is challenging because sampling over such spaces can be difficult.
Energy Discrepancies: A Score-Independent Loss for Energy-Based Models
Energy-based models are a simple yet powerful class of probabilistic models, but their widespread adoption has been limited by the computational burden of training them.
On the Identifiability of Switching Dynamical Systems
The identifiability of latent variable models has received increasing attention due to its relevance in interpretability and out-of-distribution generalisation.
Calibrating Transformers via Sparse Gaussian Processes
Transformer models have achieved profound success in prediction tasks in a wide range of applications in natural language processing, speech recognition and computer vision.
ESD: Expected Squared Difference as a Tuning-Free Trainable Calibration Measure
Studies have shown that modern neural networks tend to be poorly calibrated due to over-confident predictions.
Scalable Infomin Learning
The task of infomin learning aims to learn a representation with high utility while being uninformative about a specified target, with the latter achieved by minimising the mutual information between the representation and the target.
Markovian Gaussian Process Variational Autoencoders
Sequential VAEs have been successfully considered for many high-dimensional time series modelling problems, with many variant models relying on discrete-time mechanisms such as recurrent neural networks (RNNs).
Repairing Neural Networks by Leaving the Right Past Behind
Prediction failures of machine learning models often arise from deficiencies in training data, such as incorrect labels, outliers, and selection biases.
Learning Neural Set Functions Under the Optimal Subset Oracle
Learning neural set functions becomes increasingly more important in many applications like product recommendation and compound selection in AI-aided drug discovery.
Causal Discovery from Conditionally Stationary Time Series
Causal discovery, i. e., inferring underlying causal relationships from observational data, has been shown to be highly challenging for AI systems.
A Principled Approach to Failure Analysis and Model Repairment: Demonstration in Medical Imaging
We suggest that the quality of the identified failure types can be validated through measuring the intra- and inter-type generalisation after fine-tuning and introduce metrics to compare different subtyping methods.
Interpreting diffusion score matching using normalizing flow
Specifically, we theoretically prove that DSM (or DSD) is equivalent to the original score matching (or Stein discrepancy) evaluated in the transformed space defined by the normalizing flow, where the diffusion matrix is the inverse of the flow's Jacobian matrix.
Sparse Uncertainty Representation in Deep Learning with Inducing Weights
Bayesian neural networks and deep ensembles represent two modern paradigms of uncertainty quantification in deep learning.
Contextual HyperNetworks for Novel Feature Adaptation
While deep learning has obtained state-of-the-art results in many applications, the adaptation of neural network architectures to incorporate new output features remains a challenge, as neural networks are commonly trained to produce a fixed output dimension.
Active Slices for Sliced Stein Discrepancy
First, we provide theoretical results stating that the requirement of using optimal slicing directions in the kernelized version of SSD can be relaxed, validating the resulting discrepancy with finite random slicing directions.
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification
Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP.
Reinforcement Learning with Efficient Active Feature Acquisition
Solving real-life sequential decision making problems under partial observability involves an exploration-exploitation problem.
A Study on Efficiency in Continual Learning Inspired by Human Learning
We study a setting where the pruning phase is given a time budget, and identify connections between iterative pruning and multiple sleep cycles in humans.
Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoders
It has been long known that sparsity is an effective inductive bias for learning efficient representation of data in vectors with fixed dimensionality, and it has been explored in many areas of representation learning.
Interpreting Spatially Infinite Generative Models
In this paper we provide a firm theoretical interpretation for infinite spatial generation, by drawing connections to spatial stochastic processes.
Meta-Learning Divergences of Variational Inference
Variational inference (VI) plays an essential role in approximate Bayesian inference due to its computational efficiency and broad applicability.
Sliced Kernelized Stein Discrepancy
Kernelized Stein discrepancy (KSD), though being extensively used in goodness-of-fit tests and model learning, suffers from the curse-of-dimensionality.
A Causal View on Robustness of Neural Networks
We present a causal view on the robustness of neural networks against input manipulations, which applies not only to traditional classification tasks but also to general measurement data.
Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data
In this paper we introduce the first scalable training technique for 3D generative models from 2D data which utilizes an off-the-shelf non-differentiable renderer.
Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
We discuss those differences and propose modifications to existing regularization techniques in order to better adapt them to RL.
HM-VAEs: a Deep Generative Model for Real-valued Data with Heterogeneous Marginals
In this paper, we focused on improving VAEs for real-valued data that has heterogeneous marginal distributions.
On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation
While the explicit constraint naturally avoids posterior collapse, we use it to further understand the significance of the KL term in controlling the information transmitted through the VAE channel.
Meta-Learning for Variational Inference
Variational inference (VI) plays an essential role in approximate Bayesian inference due to its computational efficiency and general applicability.
On the Expressiveness of Approximate Inference in Bayesian Neural Networks
While Bayesian neural networks (BNNs) hold the promise of being flexible, well-calibrated statistical models, inference often requires approximations whose consequences are poorly understood.
'In-Between' Uncertainty in Bayesian Neural Networks
We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks.
Bayesian Learning for Neural Dependency Parsing
While neural dependency parsers provide state-of-the-art accuracy for several languages, they still rely on large amounts of costly labeled training data.
Interpretable Outcome Prediction with Sparse Bayesian Neural Networks in Intensive Care
However, flexible tools such as artificial neural networks (ANNs) suffer from a lack of interpretability limiting their acceptability to clinicians.
Meta-Learning for Stochastic Gradient MCMC
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling.
Variational Implicit Processes
We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables.
Disentangled Sequential Autoencoder
This architecture gives us partial control over generating content and dynamics by conditioning on either one of these sets of features.
Are Generative Classifiers More Robust to Adversarial Attacks?
There is a rising interest in studying the robustness of deep neural network classifiers against adversaries, with both advanced attack and defence techniques being actively developed.
Variational Continual Learning
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks.
Gradient Estimators for Implicit Models
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research.
Dropout Inference in Bayesian Neural Networks with Alpha-divergences
To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed.
Approximate Inference with Amortised MCMC
We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler.
Deep Gaussian Processes for Regression using Approximate Expectation Propagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers.
Rényi Divergence Variational Inference
This paper introduces the variational R\'enyi bound (VR) that extends traditional variational inference to R\'enyi's alpha-divergences.
Training Deep Gaussian Processes using Stochastic Expectation Propagation and Probabilistic Backpropagation
Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers.
Stochastic Expectation Propagation for Large Scale Gaussian Process Classification
A method for large scale Gaussian process classification has been recently proposed based on expectation propagation (EP).
Black-box $α$-divergence Minimization
Black-box alpha (BB-$\alpha$) is a new approximate inference method based on the minimization of $\alpha$-divergences.
Stochastic Expectation Propagation
Expectation propagation (EP) is a deterministic approximation algorithm that is often used to perform approximate Bayesian parameter learning.
Generating ordered list of Recommended Items: a Hybrid Recommender System of Microblog
Precise recommendation of followers helps in improving the user experience and maintaining the prosperity of twitter and microblog platforms.
