Search Results for author:
Found 34 papers, 15 papers with code
Kolmogorov n-Widths for Multitask Physics-Informed Machine Learning (PIML) Methods: Towards Robust Metrics
This topic encompasses a broad array of methods and models aimed at solving a single or a collection of PDE problems, called multitask learning.
Diffeomorphic Measure Matching with Kernels for Generative Modeling
This article presents a general framework for the transport of probability measures towards minimum divergence generative modeling and sampling using ordinary differential equations (ODEs) and Reproducing Kernel Hilbert Spaces (RKHSs), inspired by ideas from diffeomorphic matching and image registration.
Computational Hypergraph Discovery, a Gaussian Process framework for connecting the dots
Type 1: Approximate an unknown function given input/output data.
Bridging Algorithmic Information Theory and Machine Learning: A New Approach to Kernel Learning
Machine Learning (ML) and Algorithmic Information Theory (AIT) look at Complexity from different points of view.
Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEs
We introduce a priori Sobolev-space error estimates for the solution of nonlinear, and possibly parametric, PDEs using Gaussian process and kernel based methods.
Kernel Methods are Competitive for Operator Learning
We present a general kernel-based framework for learning operators between Banach spaces along with a priori error analysis and comprehensive numerical comparisons with popular neural net (NN) approaches such as Deep Operator Net (DeepONet) [Lu et al.] and Fourier Neural Operator (FNO) [Li et al.].
Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes
The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices.
Learning Dynamical Systems from Data: A Simple Cross-Validation Perspective, Part V: Sparse Kernel Flows for 132 Chaotic Dynamical Systems
In this paper, we introduce the method of \emph{Sparse Kernel Flows } in order to learn the ``best'' kernel by starting from a large dictionary of kernels.
Multiclass classification utilising an estimated algorithmic probability prior
Here we explore how algorithmic information theory, especially algorithmic probability, may aid in a machine learning task.
One-Shot Learning of Stochastic Differential Equations with Data Adapted Kernels
(2) Complete the graph (approximate unknown functions and random variables) via Maximum a Posteriori Estimation (given the data) with Gaussian Process (GP) priors on the unknown functions.
Gaussian Process Hydrodynamics
As in Smoothed Particle Hydrodynamics (SPH), GPH is a Lagrangian particle-based approach involving the tracking of a finite number of particles transported by the flow.
Learning "best" kernels from data in Gaussian process regression. With application to aerodynamics
This paper introduces algorithms to select/design kernels in Gaussian process regression/kriging surrogate modeling techniques.
Aggregation of Pareto optimal models
Under these four steps, we show that all rational/consistent aggregation rules are as follows: Give each individual Pareto optimal model a weight, introduce a weak order/ranking over the set of Pareto optimal models, aggregate a finite set of models S as the model associated with the prior obtained as the weighted average of the priors of the highest-ranked models in S. This result shows that all rational/consistent aggregation rules must follow a generalization of hierarchical Bayesian modeling.
Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-Sampled Time Series
A simple and interpretable way to learn a dynamical system from data is to interpolate its vector-field with a kernel.
Aggregation of Models, Choices, Beliefs, and Preferences
In this paper, we show that all rational aggregation rules are of the form (3).
Computational Graph Completion
The underlying problem could therefore also be interpreted as a generalization of that of solving linear systems of equations to that of approximating unknown variables and functions with noisy, incomplete, and nonlinear dependencies.
Uncertainty Quantification of the 4th kind; optimal posterior accuracy-uncertainty tradeoff with the minimum enclosing ball
Although (C) leads to the identification of an optimal prior, its approximation suffers from the curse of dimensionality and the notion of risk is one that is averaged with respect to the distribution of the data.
Solving and Learning Nonlinear PDEs with Gaussian Processes
The main idea of our method is to approximate the solution of a given PDE as the maximum a posteriori (MAP) estimator of a Gaussian process conditioned on solving the PDE at a finite number of collocation points.
Decision Theoretic Bootstrapping
The design and testing of supervised machine learning models combine two fundamental distributions: (1) the training data distribution (2) the testing data distribution.
Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods
Modeling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems.
Do ideas have shape? Idea registration as the continuous limit of artificial neural networks
We show that ResNets (and their GP generalization) converge, in the infinite depth limit, to a generalization of image registration variational algorithms.
Learning dynamical systems from data: a simple cross-validation perspective
Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems.
Competitive Mirror Descent
Finally, we obtain the next iterate by following this direction according to the dual geometry induced by the Bregman potential.
Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation
The purpose of this paper is to study two paradigms of learning hierarchical parameters: one is from the probabilistic Bayesian perspective, in particular, the empirical Bayes approach that has been largely used in Bayesian statistics; the other is from the deterministic and approximation theoretic view, and in particular the kernel flow algorithm that was proposed recently in the machine learning literature.
Sparse Cholesky factorization by Kullback-Leibler minimization
We propose to compute a sparse approximate inverse Cholesky factor $L$ of a dense covariance matrix $\Theta$ by minimizing the Kullback-Leibler divergence between the Gaussian distributions $\mathcal{N}(0, \Theta)$ and $\mathcal{N}(0, L^{-\top} L^{-1})$, subject to a sparsity constraint.
Numerical Analysis Numerical Analysis Optimization and Control Statistics Theory Computation Statistics Theory
Deep regularization and direct training of the inner layers of Neural Networks with Kernel Flows
We introduce a new regularization method for Artificial Neural Networks (ANNs) based on Kernel Flows (KFs).
Ranked #1 on
Image Classification
on QMNIST
Kernel Mode Decomposition and programmable/interpretable regression networks
Mode decomposition is a prototypical pattern recognition problem that can be addressed from the (a priori distinct) perspectives of numerical approximation, statistical inference and deep learning.
Kernel Flows: from learning kernels from data into the abyss
Kriging offers a solution to this problem based on the prior specification of a kernel.
Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity
This block-factorisation can provably be obtained in complexity $\mathcal{O} ( N \log( N ) \log^{d}( N /\epsilon) )$ in space and $\mathcal{O} ( N \log^{2}( N ) \log^{2d}( N /\epsilon) )$ in time.
Numerical Analysis Computational Complexity Data Structures and Algorithms Probability 65F30, 42C40, 65F50, 65N55, 65N75, 60G42, 68Q25, 68W40
Universal Scalable Robust Solvers from Computational Information Games and fast eigenspace adapted Multiresolution Analysis
When the solution space is a Banach space $B$ endowed with a quadratic norm $\|\cdot\|$, the optimal measure (mixed strategy) for such games (e. g. the adversarial recovery of $u\in B$, given partial measurements $[\phi_i, u]$ with $\phi_i\in B^*$, using relative error in $\|\cdot\|$-norm as a loss) is a centered Gaussian field $\xi$ solely determined by the norm $\|\cdot\|$, whose conditioning (on measurements) produces optimal bets.
Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients
Implicit schemes are popular methods for the integration of time dependent PDEs such as hyperbolic and parabolic PDEs.
Towards Machine Wald
The past century has seen a steady increase in the need of estimating and predicting complex systems and making (possibly critical) decisions with limited information.
Multigrid with rough coefficients and Multiresolution operator decomposition from Hierarchical Information Games
The resulting elementary gambles form a hierarchy of (deterministic) basis functions of $H^1_0(\Omega)$ (gamblets) that (1) are orthogonal across subscales/subbands with respect to the scalar product induced by the energy norm of the PDE (2) enable sparse compression of the solution space in $H^1_0(\Omega)$ (3) induce an orthogonal multiresolution operator decomposition.
Stochastic Variational Integrators
This paper presents a continuous and discrete Lagrangian theory for stochastic Hamiltonian systems on manifolds.
Probability 65Cxx; 37Jxx
