Search Results for author:
Found 30 papers, 9 papers with code
Mixture of Experts Soften the Curse of Dimensionality in Operator Learning
In this paper, we construct a mixture of neural operators (MoNOs) between function spaces whose complexity is distributed over a network of expert neural operators (NOs), with each NO satisfying parameter scaling restrictions.
Tighter Generalization Bounds on Digital Computers via Discrete Optimal Transport
Notably, $c_{m}\in \mathcal{O}(\sqrt{m})$ for learning models on discretized Euclidean domains.
Breaking the Curse of Dimensionality with Distributed Neural Computation
This improves the optimal bounds for traditional non-distributed deep learning models, namely ReLU MLPs, which need $\mathcal{O}(\varepsilon^{-n/2})$ parameters to achieve the same accuracy.
Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum
We derive new bounds for the condition number of kernel matrices, which we then use to enhance existing non-asymptotic test error bounds for kernel ridgeless regression in the over-parameterized regime for a fixed input dimension.
Deep Kalman Filters Can Filter
Deep Kalman filters (DKFs) are a class of neural network models that generate Gaussian probability measures from sequential data.
Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries
Furthermore, when the latent graph can be represented in the feature space of a sufficiently regular kernel, we show that the combined neural snowflake and MLP encoder do not succumb to the curse of dimensionality by using only a low-degree polynomial number of parameters in the number of nodes.
Energy-Guided Continuous Entropic Barycenter Estimation for General Costs
Optimal transport (OT) barycenters are a mathematically grounded way of averaging probability distributions while capturing their geometric properties.
Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing
We propose an optimal iterative scheme for federated transfer learning, where a central planner has access to datasets ${\cal D}_1,\dots,{\cal D}_N$ for the same learning model $f_{\theta}$.
Capacity Bounds for Hyperbolic Neural Network Representations of Latent Tree Structures
We find that the network complexity of HNN implementing the graph representation is independent of the representation fidelity/distortion.
An Approximation Theory for Metric Space-Valued Functions With A View Towards Deep Learning
Motivated by the developing mathematics of deep learning, we build universal functions approximators of continuous maps between arbitrary Polish metric spaces $\mathcal{X}$ and $\mathcal{Y}$ using elementary functions between Euclidean spaces as building blocks.
Generative Ornstein-Uhlenbeck Markets via Geometric Deep Learning
We consider the problem of simultaneously approximating the conditional distribution of market prices and their log returns with a single machine learning model.
Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation
We conclude by proposing a hypernetwork version of the subfamily of NOs as a surrogate model for the mentioned forward operator.
Instance-Dependent Generalization Bounds via Optimal Transport
Existing generalization bounds fail to explain crucial factors that drive the generalization of modern neural networks.
Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis
Causal operators (CO), such as various solution operators to stochastic differential equations, play a central role in contemporary stochastic analysis; however, there is still no canonical framework for designing Deep Learning (DL) models capable of approximating COs.
Small Transformers Compute Universal Metric Embeddings
We derive embedding guarantees for feature maps implemented by small neural networks called \emph{probabilistic transformers}.
Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?
Our first main result transcribes this "structured" approximation problem into a universality problem.
Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer
Several problems in stochastic analysis are defined through their geometry, and preserving that geometric structure is essential to generating meaningful predictions.
Universal Approximation Under Constraints is Possible with Transformers
Many practical problems need the output of a machine learning model to satisfy a set of constraints, $K$.
Universal Regular Conditional Distributions
The first strategy builds functions in $C(\mathbb{R}^d,\mathcal{P}_1(\mathbb{R}^D))$ which can be efficiently approximated by a PT, uniformly on any given compact subset of $\mathbb{R}^d$.
Universal Approximation Theorems for Differentiable Geometric Deep Learning
We show that our GDL models can approximate any continuous target function uniformly on compact sets of a controlled maximum diameter.
Optimizing Optimizers: Regret-optimal gradient descent algorithms
The need for fast and robust optimization algorithms are of critical importance in all areas of machine learning.
Learning Sub-Patterns in Piecewise Continuous Functions
Most stochastic gradient descent algorithms can optimize neural networks that are sub-differentiable in their parameters; however, this implies that the neural network's activation function must exhibit a degree of continuity which limits the neural network model's uniform approximation capacity to continuous functions.
A Canonical Transform for Strengthening the Local $L^p$-Type Universal Approximation Property
The transformed model class, denoted by $\mathscr{F}\text{-tope}$, is shown to be dense in $L^p_{\mu,\text{strict}}(\mathbb{R}^d,\mathbb{R}^D)$ which is a topological space whose elements are locally $p$-integrable functions and whose topology is much finer than usual norm topology on $L^p_{\mu}(\mathbb{R}^d,\mathbb{R}^D)$; here $\mu$ is any suitable $\sigma$-finite Borel measure $\mu$ on $\mathbb{R}^d$.
Non-Euclidean Universal Approximation
Our result is also used to show that the common practice of randomizing all but the last two layers of a DNN produces a universal family of functions with probability one.
Denise: Deep Robust Principal Component Analysis for Positive Semidefinite Matrices
The robust PCA of covariance matrices plays an essential role when isolating key explanatory features.
The Universal Approximation Property
The universal approximation property of various machine learning models is currently only understood on a case-by-case basis, limiting the rapid development of new theoretically justified neural network architectures and blurring our understanding of our current models' potential.
Partial Uncertainty and Applications to Risk-Averse Valuation
When the random vector represents the payoff of derivative security in a complete financial market, its R-conditioning with respect to the risk-neutral measure is interpreted as its risk-averse value.
NEU: A Meta-Algorithm for Universal UAP-Invariant Feature Representation
We quantify the number of parameters required for this new architecture to memorize any set of input-output pairs while simultaneously fixing every point of the input space lying outside some compact set, and we quantify the size of this set as a function of our model's depth.
Non-Euclidean Conditional Expectation and Filtering
A non-Euclidean generalization of conditional expectation is introduced and characterized as the minimizer of expected intrinsic squared-distance from a manifold-valued target.
Deep Learning in a Generalized HJM-type Framework Through Arbitrage-Free Regularization
We introduce a regularization approach to arbitrage-free factor-model selection.
