Search Results for author:
Found 67 papers, 27 papers with code
Equivalence of distance-based and RKHS-based statistics in hypothesis testing
We provide a unifying framework linking two classes of statistics used in two-sample and independence testing: on the one hand, the energy distances and distance covariances from the statistics literature; on the other, maximum mean discrepancies (MMD), that is, distances between embeddings of distributions to reproducing kernel Hilbert spaces (RKHS), as established in machine learning.
Optimal kernel choice for large-scale two-sample tests
A means of parameter selection for the two-sample test based on the MMD is proposed.
A Kernel Test for Three-Variable Interactions
We introduce kernel nonparametric tests for Lancaster three-variable interaction and for total independence, using embeddings of signed measures into a reproducing kernel Hilbert space.
Kernel Adaptive Metropolis-Hastings
A Kernel Adaptive Metropolis-Hastings algorithm is introduced, for the purpose of sampling from a target distribution with strongly nonlinear support.
A Wild Bootstrap for Degenerate Kernel Tests
A wild bootstrap method for nonparametric hypothesis tests based on kernel distribution embeddings is proposed.
Unbiased Bayes for Big Data: Paths of Partial Posteriors
A key quantity of interest in Bayesian inference are expectations of functions with respect to a posterior distribution.
K2-ABC: Approximate Bayesian Computation with Kernel Embeddings
Complicated generative models often result in a situation where computing the likelihood of observed data is intractable, while simulating from the conditional density given a parameter value is relatively easy.
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages
We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output.
Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive MCMC algorithm based on Hamiltonian Monte Carlo (HMC).
Fast Two-Sample Testing with Analytic Representations of Probability Measures
The new tests are consistent against a larger class of alternatives than the previous linear-time tests based on the (non-smoothed) empirical characteristic functions, while being much faster than the current state-of-the-art quadratic-time kernel-based or energy distance-based tests.
Kernel Sequential Monte Carlo
As KSMC does not require access to target gradients, it is particularly applicable on targets whose gradients are unknown or prohibitively expensive.
Probabilistic Integration: A Role in Statistical Computation?
A research frontier has emerged in scientific computation, wherein numerical error is regarded as a source of epistemic uncertainty that can be modelled.
DR-ABC: Approximate Bayesian Computation with Kernel-Based Distribution Regression
Approximate Bayesian computation (ABC) is an inference framework that constructs an approximation to the true likelihood based on the similarity between the observed and simulated data as measured by a predefined set of summary statistics.
Bayesian Learning of Kernel Embeddings
The posterior mean of our model is closely related to recently proposed shrinkage estimators for kernel mean embeddings, while the posterior uncertainty is a new, interesting feature with various possible applications.
Hyperspectral Image Classification with Support Vector Machines on Kernel Distribution Embeddings
We propose a novel approach for pixel classification in hyperspectral images, leveraging on both the spatial and spectral information in the data.
Large-Scale Kernel Methods for Independence Testing
Representations of probability measures in reproducing kernel Hilbert spaces provide a flexible framework for fully nonparametric hypothesis tests of independence, which can capture any type of departure from independence, including nonlinear associations and multivariate interactions.
Poisson intensity estimation with reproducing kernels
However, we prove that the representer theorem does hold in an appropriately transformed RKHS, guaranteeing that the optimization of the penalized likelihood can be cast as a tractable finite-dimensional problem.
Detecting causal associations in large nonlinear time series datasets
Detecting causal associations in time series datasets is a key challenge for novel insights into complex dynamical systems such as the Earth system or the human brain.
Methodology Atmospheric and Oceanic Physics Applications
Testing and Learning on Distributions with Symmetric Noise Invariance
Kernel embeddings of distributions and the Maximum Mean Discrepancy (MMD), the resulting distance between distributions, are useful tools for fully nonparametric two-sample testing and learning on distributions.
Bayesian Approaches to Distribution Regression
Distribution regression has recently attracted much interest as a generic solution to the problem of supervised learning where labels are available at the group level, rather than at the individual level.
Spatial Mapping with Gaussian Processes and Nonstationary Fourier Features
The use of covariance kernels is ubiquitous in the field of spatial statistics.
Causal Inference via Kernel Deviance Measures
Discovering the causal structure among a set of variables is a fundamental problem in many areas of science.
Variational Learning on Aggregate Outputs with Gaussian Processes
While a typical supervised learning framework assumes that the inputs and the outputs are measured at the same levels of granularity, many applications, including global mapping of disease, only have access to outputs at a much coarser level than that of the inputs.
Hamiltonian Variational Auto-Encoder
However, for this methodology to be practically efficient, it is necessary to obtain low-variance unbiased estimators of the ELBO and its gradients with respect to the parameters of interest.
Towards A Unified Analysis of Random Fourier Features
We study both the standard random Fourier features method for which we improve the existing bounds on the number of features required to guarantee the corresponding minimax risk convergence rate of kernel ridge regression, as well as a data-dependent modification which samples features proportional to \emph{ridge leverage scores} and further reduces the required number of features.
Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
This paper is an attempt to bridge the conceptual gaps between researchers working on the two widely used approaches based on positive definite kernels: Bayesian learning or inference using Gaussian processes on the one side, and frequentist kernel methods based on reproducing kernel Hilbert spaces on the other.
A Differentially Private Kernel Two-Sample Test
As a result, a simple chi-squared test is obtained, where a test statistic depends on a mean and covariance of empirical differences between the samples, which we perturb for a privacy guarantee.
Hyperparameter Learning via Distributional Transfer
Bayesian optimisation is a popular technique for hyperparameter learning but typically requires initial exploration even in cases where similar prior tasks have been solved.
Rejoinder for "Probabilistic Integration: A Role in Statistical Computation?"
This article is the rejoinder for the paper "Probabilistic Integration: A Role in Statistical Computation?"
Infinitely Deep Infinite-Width Networks
Infinite-width neural networks have been extensively used to study the theoretical properties underlying the extraordinary empirical success of standard, finite-width neural networks.
Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings
Current meta-learning approaches focus on learning functional representations of relationships between variables, i. e. on estimating conditional expectations in regression.
A kernel- and optimal transport- based test of independence between covariates and right-censored lifetimes
We propose a nonparametric test of independence, termed optHSIC, between a covariate and a right-censored lifetime.
Kernel Dependence Regularizers and Gaussian Processes with Applications to Algorithmic Fairness
We present a regularization approach to this problem that trades off predictive accuracy of the learned models (with respect to biased labels) for the fairness in terms of statistical parity, i. e. independence of the decisions from the sensitive covariates.
Detecting anthropogenic cloud perturbations with deep learning
One of the most pressing questions in climate science is that of the effect of anthropogenic aerosol on the Earth's energy balance.
A kernel log-rank test of independence for right-censored data
We introduce a general non-parametric independence test between right-censored survival times and covariates, which may be multivariate.
Large Scale Tensor Regression using Kernels and Variational Inference
We outline an inherent weakness of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this weakness.
Spectral Ranking with Covariates
We consider spectral approaches to the problem of ranking n players given their incomplete and noisy pairwise comparisons, but revisit this classical problem in light of player covariate information.
Learning Inconsistent Preferences with Gaussian Processes
We revisit widely used preferential Gaussian processes by Chu et al.(2005) and challenge their modelling assumption that imposes rankability of data items via latent utility function values.
A Perspective on Gaussian Processes for Earth Observation
Earth observation (EO) by airborne and satellite remote sensing and in-situ observations play a fundamental role in monitoring our planet.
Meta Learning for Causal Direction
Based on recent developments in meta learning as well as in causal inference, we introduce a novel generative model that allows distinguishing cause and effect in the small data setting.
Variational Inference with Continuously-Indexed Normalizing Flows
Continuously-indexed flows (CIFs) have recently achieved improvements over baseline normalizing flows on a variety of density estimation tasks.
Benign Overfitting and Noisy Features
Modern machine learning often operates in the regime where the number of parameters is much higher than the number of data points, with zero training loss and yet good generalization, thereby contradicting the classical bias-variance trade-off.
Kernel-based Graph Learning from Smooth Signals: A Functional Viewpoint
In this paper, we propose a novel graph learning framework that incorporates the node-side and observation-side information, and in particular the covariates that help to explain the dependency structures in graph signals.
Inter-domain Deep Gaussian Processes
We propose Inter-domain Deep Gaussian Processes, an extension of inter-domain shallow GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs), and demonstrate how to leverage existing approximate inference methods to perform simple and scalable approximate inference using inter-domain features in DGPs.
Survival Regression with Proper Scoring Rules and Monotonic Neural Networks
We consider frequently used scoring rules for right-censored survival regression models such as time-dependent concordance, survival-CRPS, integrated Brier score and integrated binomial log-likelihood, and prove that neither of them is a proper scoring rule.
Deconditional Downscaling with Gaussian Processes
Yet, when LR samples are modeled as aggregate conditional means of HR samples with respect to a mediating variable that is globally observed, the recovery of the underlying fine-grained field can be framed as taking an "inverse" of the conditional expectation, namely a deconditioning problem.
Connections and Equivalences between the Nyström Method and Sparse Variational Gaussian Processes
We investigate the connections between sparse approximation methods for making kernel methods and Gaussian processes (GPs) scalable to large-scale data, focusing on the Nystr\"om method and the Sparse Variational Gaussian Processes (SVGP).
BayesIMP: Uncertainty Quantification for Causal Data Fusion
While causal models are becoming one of the mainstays of machine learning, the problem of uncertainty quantification in causal inference remains challenging.
RKHS-SHAP: Shapley Values for Kernel Methods
Feature attribution for kernel methods is often heuristic and not individualised for each prediction.
A Kernel Test for Causal Association via Noise Contrastive Backdoor Adjustment
Causal inference grows increasingly complex as the number of confounders increases.
Identifying Pauli spin blockade using deep learning
Pauli spin blockade (PSB) can be employed as a great resource for spin qubit initialisation and readout even at elevated temperatures but it can be difficult to identify.
Giga-scale Kernel Matrix Vector Multiplication on GPU
Kernel matrix-vector multiplication (KMVM) is a foundational operation in machine learning and scientific computing.
Selection, Ignorability and Challenges With Causal Fairness
In order to achieve this, we must have causal models that are able to capture what someone would be like if we were to counterfactually change these traits.
AODisaggregation: toward global aerosol vertical profiles
In this work, we develop a framework for the vertical disaggregation of AOD into extinction profiles, i. e. the measure of light extinction throughout an atmospheric column, using readily available vertically resolved meteorological predictors such as temperature, pressure or relative humidity.
Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learning
We develop a framework for generalized variational inference in infinite-dimensional function spaces and use it to construct a method termed Gaussian Wasserstein inference (GWI).
Explaining Preferences with Shapley Values
While preference modelling is becoming one of the pillars of machine learning, the problem of preference explanation remains challenging and underexplored.
Discussion of `Multiscale Fisher's Independence Test for Multivariate Dependence'
We discuss how MultiFIT, the Multiscale Fisher's Independence Test for Multivariate Dependence proposed by Gorsky and Ma (2022), compares to existing linear-time kernel tests based on the Hilbert-Schmidt independence criterion (HSIC).
Kernel Biclustering algorithm in Hilbert Spaces
Biclustering algorithms partition data and covariates simultaneously, providing new insights in several domains, such as analyzing gene expression to discover new biological functions.
Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddings
The problem of sequentially maximizing the expectation of a function seeks to maximize the expected value of a function of interest without having direct control on its features.
Bayesian Counterfactual Mean Embeddings and Off-Policy Evaluation
The framework naturally extends to the setting where one observes multiple treatment effects (e. g. an intermediate effect after an interim period, and an ultimate treatment effect which is of main interest) and allows for additionally modelling uncertainty about the relationship of these effects.
Doubly Robust Kernel Statistics for Testing Distributional Treatment Effects
These improved estimators are inspired by doubly robust estimators of the causal mean, using a similar form within the kernel space.
Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge
A directed acyclic graph (DAG) provides valuable prior knowledge that is often discarded in regression tasks in machine learning.
A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods
We establish the first mathematically rigorous link between Bayesian, variational Bayesian, and ensemble methods.
FaIRGP: A Bayesian Energy Balance Model for Surface Temperatures Emulation
The result is an emulator that \textit{(i)} enjoys the flexibility of statistical machine learning models and can learn from data, and \textit{(ii)} has a robust physical grounding with interpretable parameters that can be used to make inference about the climate system.
Exact, Fast and Expressive Poisson Point Processes via Squared Neural Families
Maximum likelihood and maximum a posteriori estimates in a reparameterisation of the final layer of the intensity function can be obtained by solving a (strongly) convex optimisation problem using projected gradient descent.
Neural-Kernel Conditional Mean Embeddings
In conditional density estimation tasks, our NN-CME hybrid achieves competitive performance and often surpasses existing deep learning-based methods.
Inter-domain Deep Gaussian Processes with RKHS Fourier Features
We propose Inter-domain Deep Gaussian Processes with RKHS Fourier Features, an extension of shallow inter-domain GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs) and demonstrate how to leverage existing approximate inference approaches to perform simple and scalable approximate inference on Inter-domain Deep Gaussian Processes.
