Search Results for author:
Found 56 papers, 22 papers with code
Closed-form Filtering for Non-linear Systems
Sequential Bayesian Filtering aims to estimate the current state distribution of a Hidden Markov Model, given the past observations.
A PAC-Bayesian Link Between Generalisation and Flat Minima
Modern machine learning usually involves predictors in the overparametrised setting (number of trained parameters greater than dataset size), and their training yield not only good performances on training data, but also good generalisation capacity.
Tighter Generalisation Bounds via Interpolation
We also instantiate our bounds as training objectives, yielding non-trivial guarantees and practical performances.
A note on regularised NTK dynamics with an application to PAC-Bayesian training
We establish explicit dynamics for neural networks whose training objective has a regularising term that constrains the parameters to remain close to their initial value.
Federated Learning with Nonvacuous Generalisation Bounds
We introduce a novel strategy to train randomised predictors in federated learning, where each node of the network aims at preserving its privacy by releasing a local predictor but keeping secret its training dataset with respect to the other nodes.
Comparing Comparators in Generalization Bounds
We derive generic information-theoretic and PAC-Bayesian generalization bounds involving an arbitrary convex comparator function, which measures the discrepancy between the training and population loss.
Generalization Bounds: Perspectives from Information Theory and PAC-Bayes
Over the past decades, the PAC-Bayesian approach has been established as a flexible framework to address the generalization capabilities of machine learning algorithms, and design new ones.
Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisation
PAC-Bayes learning is an established framework to both assess the generalisation ability of learning algorithms, and design new learning algorithm by exploiting generalisation bounds as training objectives.
Optimistically Tempered Online Learning
Optimistic Online Learning algorithms have been developed to exploit expert advices, assumed optimistically to be always useful.
Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty
We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds.
PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingales
While PAC-Bayes is now an established learning framework for light-tailed losses (\emph{e. g.}, subgaussian or subexponential), its extension to the case of heavy-tailed losses remains largely uncharted and has attracted a growing interest in recent years.
Generalisation under gradient descent via deterministic PAC-Bayes
We establish disintegrated PAC-Bayesian generalisation bounds for models trained with gradient descent methods or continuous gradient flows.
Efficient Aggregated Kernel Tests using Incomplete $U$-statistics
We derive non-asymptotic uniform separation rates for MMDAggInc and HSICAggInc, and quantify exactly the trade-off between computational efficiency and the attainable rates: this result is novel for tests based on incomplete $U$-statistics, to our knowledge.
On Margins and Generalisation for Voting Classifiers
We study the generalisation properties of majority voting on finite ensembles of classifiers, proving margin-based generalisation bounds via the PAC-Bayes theory.
Online PAC-Bayes Learning
Most PAC-Bayesian bounds hold in the batch learning setting where data is collected at once, prior to inference or prediction.
Reprint: a randomized extrapolation based on principal components for data augmentation
This paper presents REPRINT, a simple and effective hidden-space data augmentation method for imbalanced data classification.
On PAC-Bayesian reconstruction guarantees for VAEs
Despite its wide use and empirical successes, the theoretical understanding and study of the behaviour and performance of the variational autoencoder (VAE) have only emerged in the past few years.
Controlling Multiple Errors Simultaneously with a PAC-Bayes Bound
Current PAC-Bayes generalisation bounds are restricted to scalar metrics of performance, such as the loss or error rate.
Measuring dissimilarity with diffeomorphism invariance
Measures of similarity (or dissimilarity) are a key ingredient to many machine learning algorithms.
On change of measure inequalities for $f$-divergences
We propose new change of measure inequalities based on $f$-divergences (of which the Kullback-Leibler divergence is a particular case).
Non-Vacuous Generalisation Bounds for Shallow Neural Networks
We focus on a specific class of shallow neural networks with a single hidden layer, namely those with $L_2$-normalised data and either a sigmoid-shaped Gaussian error function ("erf") activation or a Gaussian Error Linear Unit (GELU) activation.
KSD Aggregated Goodness-of-fit Test
KSDAgg avoids splitting the data to perform kernel selection (which leads to a loss in test power), and rather maximises the test power over a collection of kernels.
Progress in Self-Certified Neural Networks
We then show that in data starvation regimes, holding out data for the test set bounds adversely affects generalisation performance, while self-certified strategies based on PAC-Bayes bounds do not suffer from this drawback, proving that they might be a suitable choice for the small data regime.
MMD Aggregated Two-Sample Test
In practice, this parameter is unknown and, hence, the optimal MMD test with this particular kernel cannot be used.
Learning PAC-Bayes Priors for Probabilistic Neural Networks
We experiment on 6 datasets with different strategies and amounts of data to learn data-dependent PAC-Bayes priors, and we compare them in terms of their effect on test performance of the learnt predictors and tightness of their risk certificate.
On Margins and Derandomisation in PAC-Bayes
We give a general recipe for derandomising PAC-Bayesian bounds using margins, with the critical ingredient being that our randomised predictions concentrate around some value.
Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound
We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties.
Upper and Lower Bounds on the Performance of Kernel PCA
Principal Component Analysis (PCA) is a popular method for dimension reduction and has attracted an unfailing interest for decades.
A PAC-Bayesian Perspective on Structured Prediction with Implicit Loss Embeddings
Many practical machine learning tasks can be framed as Structured prediction problems, where several output variables are predicted and considered interdependent.
An end-to-end data-driven optimisation framework for constrained trajectories
Many real-world problems require to optimise trajectories under constraints.
Cluster-Specific Predictions with Multi-Task Gaussian Processes
A variational EM algorithm is derived for dealing with the optimisation of the hyper-parameters along with the hyper-posteriors' estimation of latent variables and processes.
Forecasting elections results via the voter model with stubborn nodes
We are able to perform time-evolving estimates of the model parameters and use these to forecast the vote shares for each party in any election.
MAGMA: Inference and Prediction with Multi-Task Gaussian Processes
A novel multi-task Gaussian process (GP) framework is proposed, by using a common mean process for sharing information across tasks.
PAC-Bayesian Bound for the Conditional Value at Risk
Conditional Value at Risk (CVaR) is a family of "coherent risk measures" which generalize the traditional mathematical expectation.
Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networks
We make three related contributions motivated by the challenge of training stochastic neural networks, particularly in a PAC-Bayesian setting: (1) we show how averaging over an ensemble of stochastic neural networks enables a new class of \emph{partially-aggregated} estimators; (2) we show that these lead to provably lower-variance gradient estimates for non-differentiable signed-output networks; (3) we reformulate a PAC-Bayesian bound for these networks to derive a directly optimisable, differentiable objective and a generalisation guarantee, without using a surrogate loss or loosening the bound.
Towards control of opinion diversity by introducing zealots into a polarised social group
We explore a method to influence or even control the diversity of opinions within a polarised social group.
PAC-Bayes unleashed: generalisation bounds with unbounded losses
We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions.
From industry-wide parameters to aircraft-centric on-flight inference: improving aeronautics performance prediction with machine learning
We illustrate our approach by focusing on the estimation of the drag and lift coefficients from recorded flight data.
Kernel-Based Ensemble Learning in Python
We propose a new supervised learning algorithm, for classification and regression problems where two or more preliminary predictors are available.
PAC-Bayesian Contrastive Unsupervised Representation Learning
Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data.
Still no free lunches: the price to pay for tighter PAC-Bayes bounds
"No free lunch" results state the impossibility of obtaining meaningful bounds on the error of a learning algorithm without prior assumptions and modelling.
Online k-means Clustering
The specific formulation we use is the $k$-means objective: At each time step the algorithm has to maintain a set of k candidate centers and the loss incurred is the squared distance between the new point and the closest center.
Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
We present a comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory.
Model Validation Using Mutated Training Labels: An Exploratory Study
MV mutates training data labels, retrains the model against the mutated data, then uses the metamorphic relation that captures the consequent training performance changes to assess model fit.
Non-linear aggregation of filters to improve image denoising
We introduce a novel aggregation method to efficiently perform image denoising.
Revisiting clustering as matrix factorisation on the Stiefel manifold
This paper studies clustering for possibly high dimensional data (e. g. images, time series, gene expression data, and many other settings), and rephrase it as low rank matrix estimation in the PAC-Bayesian framework.
A Primer on PAC-Bayesian Learning
Generalised Bayesian learning algorithms are increasingly popular in machine learning, due to their PAC generalisation properties and flexibility.
Sequential Learning of Principal Curves: Summarizing Data Streams on the Fly
When confronted with massive data streams, summarizing data with dimension reduction methods such as PCA raises theoretical and algorithmic pitfalls.
Decentralized learning with budgeted network load using Gaussian copulas and classifier ensembles
We examine a network of learners which address the same classification task but must learn from different data sets.
Pycobra: A Python Toolbox for Ensemble Learning and Visualisation
We introduce \texttt{pycobra}, a Python library devoted to ensemble learning (regression and classification) and visualisation.
Simpler PAC-Bayesian Bounds for Hostile Data
In these bounds the Kullack-Leibler divergence is replaced with a general version of Csisz\'ar's $f$-divergence.
Stability revisited: new generalisation bounds for the Leave-one-Out
The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms.
A Quasi-Bayesian Perspective to Online Clustering
When faced with high frequency streams of data, clustering raises theoretical and algorithmic pitfalls.
An Oracle Inequality for Quasi-Bayesian Non-Negative Matrix Factorization
The aim of this paper is to provide some theoretical understanding of quasi-Bayesian aggregation methods non-negative matrix factorization.
PAC-Bayesian High Dimensional Bipartite Ranking
This paper is devoted to the bipartite ranking problem, a classical statistical learning task, in a high dimensional setting.
