Search Results for author:
Found 105 papers, 55 papers with code
Demystifying MMD GANs
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs.
Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
In this context, the MMD may be used in two roles: first, as a discriminator, either directly on the samples, or on features of the samples.
A Linear-Time Kernel Goodness-of-Fit Test
We propose a novel adaptive test of goodness-of-fit, with computational cost linear in the number of samples.
Interpretable Distribution Features with Maximum Testing Power
Two semimetrics on probability distributions are proposed, given as the sum of differences of expectations of analytic functions evaluated at spatial or frequency locations (i. e, features).
Generalized Energy Based Models
We show that both training stages are well-defined: the energy is learned by maximising a generalized likelihood, and the resulting energy-based loss provides informative gradients for learning the base.
Learning Deep Kernels for Non-Parametric Two-Sample Tests
We propose a class of kernel-based two-sample tests, which aim to determine whether two sets of samples are drawn from the same distribution.
Ranked #1 on
Two-sample testing
on HIGGS Data Set
BRUNO: A Deep Recurrent Model for Exchangeable Data
We present a novel model architecture which leverages deep learning tools to perform exact Bayesian inference on sets of high dimensional, complex observations.
Self-Supervised Learning with Kernel Dependence Maximization
We approach self-supervised learning of image representations from a statistical dependence perspective, proposing Self-Supervised Learning with the Hilbert-Schmidt Independence Criterion (SSL-HSIC).
On gradient regularizers for MMD GANs
We propose a principled method for gradient-based regularization of the critic of GAN-like models trained by adversarially optimizing the kernel of a Maximum Mean Discrepancy (MMD).
Ranked #128 on
Image Generation
on CIFAR-10
Kernel Adaptive Metropolis-Hastings
A Kernel Adaptive Metropolis-Hastings algorithm is introduced, for the purpose of sampling from a target distribution with strongly nonlinear support.
A Test of Relative Similarity For Model Selection in Generative Models
Probabilistic generative models provide a powerful framework for representing data that avoids the expense of manual annotation typically needed by discriminative approaches.
Learning deep kernels for exponential family densities
The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching.
Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive MCMC algorithm based on Hamiltonian Monte Carlo (HMC).
A Kernel Test of Goodness of Fit
Our test statistic is based on an empirical estimate of this divergence, taking the form of a V-statistic in terms of the log gradients of the target density and the kernel.
Efficient Conditionally Invariant Representation Learning
The procedure requires just a single ridge regression from $Y$ to kernelized features of $Z$, which can be done in advance.
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages
We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output.
Informative Features for Model Comparison
Given two candidate models, and a set of target observations, we address the problem of measuring the relative goodness of fit of the two models.
Density Estimation in Infinite Dimensional Exponential Families
When $p_0\in\mathcal{P}$, we show that the proposed estimator is consistent, and provide a convergence rate of $n^{-\min\left\{\frac{2}{3},\frac{2\beta+1}{2\beta+2}\right\}}$ in Fisher divergence under the smoothness assumption that $\log p_0\in\mathcal{R}(C^\beta)$ for some $\beta\ge 0$, where $C$ is a certain Hilbert-Schmidt operator on $H$ and $\mathcal{R}(C^\beta)$ denotes the image of $C^\beta$.
An Adaptive Test of Independence with Analytic Kernel Embeddings
The dependence measure is the difference between analytic embeddings of the joint distribution and the product of the marginals, evaluated at a finite set of locations (features).
A Kernel Independence Test for Random Processes
A new non parametric approach to the problem of testing the independence of two random process is developed.
B-tests: Low Variance Kernel Two-Sample Tests
A family of maximum mean discrepancy (MMD) kernel two-sample tests is introduced.
MMD Aggregated Two-Sample Test
In practice, this parameter is unknown and, hence, the optimal MMD test with this particular kernel cannot be used.
Large-Scale Kernel Methods for Independence Testing
Representations of probability measures in reproducing kernel Hilbert spaces provide a flexible framework for fully nonparametric hypothesis tests of independence, which can capture any type of departure from independence, including nonlinear associations and multivariate interactions.
Kernelized Wasserstein Natural Gradient
Many machine learning problems can be expressed as the optimization of some cost functional over a parametric family of probability distributions.
Exponential Family Estimation via Adversarial Dynamics Embedding
We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks.
Learning Deep Features in Instrumental Variable Regression
We propose a novel method, deep feature instrumental variable regression (DFIV), to address the case where relations between instruments, treatments, and outcomes may be nonlinear.
Efficient Wasserstein Natural Gradients for Reinforcement Learning
A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL).
Recovery of non-linear cause-effect relationships from linearly mixed neuroimaging data
Causal inference concerns the identification of cause-effect relationships between variables.
MERLiN: Mixture Effect Recovery in Linear Networks
Causal inference concerns the identification of cause-effect relationships between variables, e. g. establishing whether a stimulus affects activity in a certain brain region.
Kernel Instrumental Variable Regression
Instrumental variable (IV) regression is a strategy for learning causal relationships in observational data.
Fast Two-Sample Testing with Analytic Representations of Probability Measures
The new tests are consistent against a larger class of alternatives than the previous linear-time tests based on the (non-smoothed) empirical characteristic functions, while being much faster than the current state-of-the-art quadratic-time kernel-based or energy distance-based tests.
Maximum Mean Discrepancy Gradient Flow
We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties.
KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support
We study the gradient flow for a relaxed approximation to the Kullback-Leibler (KL) divergence between a moving source and a fixed target distribution.
Kernel Exponential Family Estimation via Doubly Dual Embedding
We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space.
KSD Aggregated Goodness-of-fit Test
KSDAgg avoids splitting the data to perform kernel selection (which leads to a loss in test power), and rather maximises the test power over a collection of kernels.
A Distributional Analogue to the Successor Representation
This paper contributes a new approach for distributional reinforcement learning which elucidates a clean separation of transition structure and reward in the learning process.
Distributional Reinforcement Learning
Model-based Reinforcement Learning
+1
Efficient Aggregated Kernel Tests using Incomplete $U$-statistics
We derive non-asymptotic uniform separation rates for MMDAggInc and HSICAggInc, and quantify exactly the trade-off between computational efficiency and the attainable rates: this result is novel for tests based on incomplete $U$-statistics, to our knowledge.
Practical Kernel Tests of Conditional Independence
We describe a data-efficient, kernel-based approach to statistical testing of conditional independence.
A Wild Bootstrap for Degenerate Kernel Tests
A wild bootstrap method for nonparametric hypothesis tests based on kernel distribution embeddings is proposed.
Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction
In particular, we provide a unifying view of two-stage and moment restriction approaches for solving this problem in a nonlinear setting.
A kernel Stein test of goodness of fit for sequential models
We propose a goodness-of-fit measure for probability densities modeling observations with varying dimensionality, such as text documents of differing lengths or variable-length sequences.
Kernel Conditional Exponential Family
A nonparametric family of conditional distributions is introduced, which generalizes conditional exponential families using functional parameters in a suitable RKHS.
A low variance consistent test of relative dependency
Such a test enables us to determine whether one source variable is significantly more dependent on a first target variable or a second.
On Instrumental Variable Regression for Deep Offline Policy Evaluation
By applying different IV techniques to OPE, we are not only able to recover previously proposed OPE methods such as model-based techniques but also to obtain competitive new techniques.
Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation
Proxy causal learning (PCL) is a method for estimating the causal effect of treatments on outcomes in the presence of unobserved confounding, using proxies (structured side information) for the confounder.
Efficient and principled score estimation with Nyström kernel exponential families
We propose a fast method with statistical guarantees for learning an exponential family density model where the natural parameter is in a reproducing kernel Hilbert space, and may be infinite-dimensional.
A kernel log-rank test of independence for right-censored data
We introduce a general non-parametric independence test between right-censored survival times and covariates, which may be multivariate.
Composite Goodness-of-fit Tests with Kernels
Model misspecification can create significant challenges for the implementation of probabilistic models, and this has led to development of a range of robust methods which directly account for this issue.
Exchangeable Models in Meta Reinforcement Learning
One recent approach to meta reinforcement learning (meta-RL) is to integrate models for task inference with models for control.
Distributional Bellman Operators over Mean Embeddings
We propose a novel algorithmic framework for distributional reinforcement learning, based on learning finite-dimensional mean embeddings of return distributions.
MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting
We propose novel statistics which maximise the power of a two-sample test based on the Maximum Mean Discrepancy (MMD), by adapting over the set of kernels used in defining it.
Model-based Kernel Sum Rule: Kernel Bayesian Inference with Probabilistic Models
Our contribution in this paper is to introduce a novel approach, termed the {\em model-based kernel sum rule} (Mb-KSR), to combine a probabilistic model and kernel Bayesian inference.
Fast Non-Parametric Tests of Relative Dependency and Similarity
The second test, called the relative test of similarity, is use to determine which of the two samples from arbitrary distributions is significantly closer to a reference sample of interest and the relative measure of similarity is based on the Maximum Mean Discrepancy (MMD).
Learning Theory for Distribution Regression
In this paper, we study a simple, analytically computable, ridge regression-based alternative to distribution regression, where we embed the distributions to a reproducing kernel Hilbert space, and learn the regressor from the embeddings to the outputs.
GP-select: Accelerating EM using adaptive subspace preselection
We propose a nonparametric procedure to achieve fast inference in generative graphical models when the number of latent states is very large.
A Kernel Test for Three-Variable Interactions with Random Processes
The main contributions of this paper are twofold: first, we prove that the Lancaster statistic satisfies the conditions required to estimate the quantiles of the null distribution using the wild bootstrap; second, the manner in which this is proved is novel, simpler than existing methods, and can further be applied to other statistics.
Kernel Mean Shrinkage Estimators
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs.
Filtering with State-Observation Examples via Kernel Monte Carlo Filter
In particular, the sampling and resampling procedures are novel in being expressed using kernel mean embeddings, so we theoretically analyze their behaviors.
Two-stage Sampled Learning Theory on Distributions
To the best of our knowledge, the only existing method with consistency guarantees for distribution regression requires kernel density estimation as an intermediate step (which suffers from slow convergence issues in high dimensions), and the domain of the distributions to be compact Euclidean.
A simpler condition for consistency of a kernel independence test
The HSIC is defined as the distance between the embedding of the joint distribution, and the embedding of the product of the marginals, in a Reproducing Kernel Hilbert Space (RKHS).
Passing Expectation Propagation Messages with Kernel Methods
We propose to learn a kernel-based message operator which takes as input all expectation propagation (EP) incoming messages to a factor node and produces an outgoing message.
Equivalence of distance-based and RKHS-based statistics in hypothesis testing
We provide a unifying framework linking two classes of statistics used in two-sample and independence testing: on the one hand, the energy distances and distance covariances from the statistics literature; on the other, maximum mean discrepancies (MMD), that is, distances between embeddings of distributions to reproducing kernel Hilbert spaces (RKHS), as established in machine learning.
Hilbert Space Embeddings of Predictive State Representations
The essence is to represent the state as a nonparametric conditional embedding operator in a Reproducing Kernel Hilbert Space (RKHS) and leverage recent work in kernel methods to estimate, predict, and update the representation.
A Kernel Test for Three-Variable Interactions
We introduce kernel nonparametric tests for Lancaster three-variable interaction and for total independence, using embeddings of signed measures into a reproducing kernel Hilbert space.
Kernel Mean Estimation and Stein's Effect
A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions.
Hilbert space embeddings and metrics on probability measures
First, we consider the question of determining the conditions on the kernel $k$ for which $\gamma_k$ is a metric: such $k$ are denoted {\em characteristic kernels}.
Antithetic and Monte Carlo kernel estimators for partial rankings
We also present a novel variance reduction scheme based on an antithetic variate construction between permutations to obtain an improved estimator for the Mallows kernel.
On integral probability metrics, φ-divergences and binary classification
First, to understand the relation between IPMs and $\phi$-divergences, the necessary and sufficient conditions under which these classes intersect are derived: the total variation distance is shown to be the only non-trivial $\phi$-divergence that is also an IPM.
Information Theory Information Theory
B-test: A Non-parametric, Low Variance Kernel Two-sample Test
We propose a family of maximum mean discrepancy (MMD) kernel two-sample tests that have low sample complexity and are consistent.
Optimal kernel choice for large-scale two-sample tests
A means of parameter selection for the two-sample test based on the MMD is proposed.
Kernel Bayes' Rule
A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on kernel representations of probabilities in reproducing kernel Hilbert spaces.
A Fast, Consistent Kernel Two-Sample Test
A kernel embedding of probability distributions into reproducing kernel Hilbert spaces (RKHS) has recently been proposed, which allows the comparison of two probability measures P and Q based on the distance between their respective embeddings: for a sufficiently rich RKHS, this distance is zero if and only if P and Q coincide.
Nonlinear directed acyclic structure learning with weakly additive noise models
This results in a more computationally efficient approach that is useful for arbitrary distributions even when additive noise models are invertible.
Learning Taxonomies by Dependence Maximization
We introduce a family of unsupervised algorithms, numerical taxonomy clustering, to simultaneously cluster data, and to learn a taxonomy that encodes the relationship between the clusters.
Characteristic Kernels on Groups and Semigroups
Embeddings of random variables in reproducing kernel Hilbert spaces (RKHSs) may be used to conduct statistical inference based on higher order moments.
Kernel Measures of Independence for non-iid Data
Many machine learning algorithms can be formulated in the framework of statistical independence such as the Hilbert Schmidt Independence Criterion.
A Kernel Stein Test for Comparing Latent Variable Models
We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the marginal distribution of the observed variables is intractable.
Counterfactual Distribution Regression for Structured Inference
The inference problem is how information concerning perturbations, with particular covariates such as location and time, can be generalized to predict the effect of novel perturbations.
Modelling transition dynamics in MDPs with RKHS embeddings
For policy optimisation we compare with least-squares policy iteration where a Gaussian process is used for value function estimation.
Deep Layer-wise Networks Have Closed-Form Weights
There is currently a debate within the neuroscience community over the likelihood of the brain performing backpropagation (BP).
A Non-Asymptotic Analysis for Stein Variational Gradient Descent
We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution $\pi\propto e^{-V}$ on $\mathbb{R}^d$.
Kernelized Stein Discrepancy Tests of Goodness-of-fit for Time-to-Event Data
Survival Analysis and Reliability Theory are concerned with the analysis of time-to-event data, in which observations correspond to waiting times until an event of interest such as death from a particular disease or failure of a component in a mechanical system.
Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves
We propose estimators based on kernel ridge regression for nonparametric causal functions such as dose, heterogeneous, and incremental response curves.
A Weaker Faithfulness Assumption based on Triple Interactions
One of the core assumptions in causal discovery is the faithfulness assumption, i. e., assuming that independencies found in the data are due to separations in the true causal graph.
Kernel Dependence Network
We propose a greedy strategy to spectrally train a deep network for multi-class classification.
A kernel test for quasi-independence
We consider settings in which the data of interest correspond to pairs of ordered times, e. g, the birth times of the first and second child, the times at which a new user creates an account and makes the first purchase on a website, and the entry and survival times of patients in a clinical trial.
A case for new neural network smoothness constraints
How sensitive should machine learning models be to input changes?
Towards an Understanding of Benign Overfitting in Neural Networks
Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss; yet surprisingly, they possess near-optimal prediction performance, contradicting classical learning theory.
Sequential Kernel Embedding for Mediated and Time-Varying Dose Response Curves
We propose simple nonparametric estimators for mediated and time-varying dose response curves based on kernel ridge regression.
A case for new neural networks smoothness constraints
How sensitive should machine learning models be to input changes?
Deep Layer-wise Networks Have Closed-Form Weights
There is currently a debate within the neuroscience community over the likelihood of the brain performing backpropagation (BP).
Importance Weighting Approach in Kernel Bayes' Rule
We study a nonparametric approach to Bayesian computation via feature means, where the expectation of prior features is updated to yield expected kernel posterior features, based on regression from learned neural net or kernel features of the observations.
Causal Inference with Treatment Measurement Error: A Nonparametric Instrumental Variable Approach
We propose a kernel-based nonparametric estimator for the causal effect when the cause is corrupted by error.
Discussion of `Multiscale Fisher's Independence Test for Multivariate Dependence'
We discuss how MultiFIT, the Multiscale Fisher's Independence Test for Multivariate Dependence proposed by Gorsky and Ma (2022), compares to existing linear-time kernel tests based on the Hilbert-Schmidt independence criterion (HSIC).
Optimal Rates for Regularized Conditional Mean Embedding Learning
We address the misspecified setting, where the target CME is in the space of Hilbert-Schmidt operators acting from an input interpolation space between $\mathcal{H}_X$ and $L_2$, to $\mathcal{H}_Y$.
A Neural Mean Embedding Approach for Back-door and Front-door Adjustment
We consider the estimation of average and counterfactual treatment effects, under two settings: back-door adjustment and front-door adjustment.
Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference
We introduce two synthetic likelihood methods for Simulation-Based Inference (SBI), to conduct either amortized or targeted inference from experimental observations when a high-fidelity simulator is available.
Controlling Moments with Kernel Stein Discrepancies
Kernel Stein discrepancies (KSDs) measure the quality of a distributional approximation and can be computed even when the target density has an intractable normalizing constant.
Adapting to Latent Subgroup Shifts via Concepts and Proxies
We show that the optimal target predictor can be non-parametrically identified with the help of concept and proxy variables available only in the source domain, and unlabeled data from the target.
Deep Hypothesis Tests Detect Clinically Relevant Subgroup Shifts in Medical Images
We formulate subgroup shift detection in the framework of statistical hypothesis testing and show that recent state-of-the-art statistical tests can be effectively applied to subgroup shift detection on medical imaging data.
Prediction under Latent Subgroup Shifts with High-Dimensional Observations
We introduce a new approach to prediction in graphical models with latent-shift adaptation, i. e., where source and target environments differ in the distribution of an unobserved confounding latent variable.
Nonlinear Meta-Learning Can Guarantee Faster Rates
Many recent theoretical works on \emph{meta-learning} aim to achieve guarantees in leveraging similar representational structures from related tasks towards simplifying a target task.
Kernel Single Proxy Control for Deterministic Confounding
We consider the problem of causal effect estimation with an unobserved confounder, where we observe a proxy variable that is associated with the confounder.
Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithm
We present the first optimal rates for infinite-dimensional vector-valued ridge regression on a continuous scale of norms that interpolate between $L_2$ and the hypothesis space, which we consider as a vector-valued reproducing kernel Hilbert space.
Proxy Methods for Domain Adaptation
We study the problem of domain adaptation under distribution shift, where the shift is due to a change in the distribution of an unobserved, latent variable that confounds both the covariates and the labels.
