Search Results for author:
Found 30 papers, 7 papers with code
On the Role of Initialization on the Implicit Bias in Deep Linear Networks
In this work, we focus on investigating the implicit bias originating from weight initialization.
Manifold Free Riemannian Optimization
In case some of the components are given analytically (e. g., if the cost function and its gradient are given explicitly, or if the tangent spaces can be computed), the algorithm can be easily adapted to use the accurate expressions instead of the approximations.
PCENet: High Dimensional Surrogate Modeling for Learning Uncertainty
In this paper, we present a novel surrogate model for representation learning and uncertainty quantification, which aims to deal with data of moderate to high dimensions.
Random Gegenbauer Features for Scalable Kernel Methods
Our proposed GZK family, generalizes the zonal kernels (i. e., dot-product kernels on the unit sphere) by introducing radial factors in their Gegenbauer series expansion, and includes a wide range of ubiquitous kernel functions such as the entirety of dot-product kernels as well as the Gaussian and the recently introduced Neural Tangent kernels.
Low-Rank Updates of Matrix Square Roots
Given a low rank perturbation to a matrix, we argue that a low-rank approximate correction to the (inverse) square root exists.
Dimensionality Reduction of Longitudinal 'Omics Data using Modern Tensor Factorization
Precision medicine is a clinical approach for disease prevention, detection and treatment, which considers each individual's genetic background, environment and lifestyle.
Scaling Neural Tangent Kernels via Sketching and Random Features
To accelerate learning with NTK, we design a near input-sparsity time approximation algorithm for NTK, by sketching the polynomial expansions of arc-cosine kernels: our sketch for the convolutional counterpart of NTK (CNTK) can transform any image using a linear runtime in the number of pixels.
Random Features for the Neural Tangent Kernel
We combine random features of the arc-cosine kernels with a sketching-based algorithm which can run in linear with respect to both the number of data points and input dimension.
Gauss-Legendre Features for Gaussian Process Regression
Our method is very much inspired by the well-known random Fourier features approach, which also builds low-rank approximations via numerical integration.
Faster Randomized Infeasible Interior Point Methods for Tall/Wide Linear Programs
Linear programming (LP) is used in many machine learning applications, such as $\ell_1$-regularized SVMs, basis pursuit, nonnegative matrix factorization, etc.
Experimental Design for Overparameterized Learning with Application to Single Shot Deep Active Learning
Unfortunately, classical theory on optimal experimental design focuses on selecting examples in order to learn underparameterized (and thus, non-interpolative) models, while modern machine learning models such as deep neural networks are overparameterized, and oftentimes are trained to be interpolative.
Dynamic Graph Convolutional Networks Using the Tensor M-Product
In recent years, a variety of graph neural networks (GNNs) have been successfully applied for representation learning and prediction on such graphs.
Polynomial Tensor Sketch for Element-wise Function of Low-Rank Matrix
This paper studies how to sketch element-wise functions of low-rank matrices.
Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold
In this paper we develop the geometric components required to perform Riemannian optimization on the generalized Stiefel manifold equipped with a non-standard metric, and illustrate theoretically and numerically the use of those components and the effect of Riemannian preconditioning for solving optimization problems on the generalized Stiefel manifold.
A Universal Sampling Method for Reconstructing Signals with Simple Fourier Transforms
We formalize this intuition by showing that, roughly, a continuous signal from a given class can be approximately reconstructed using a number of samples proportional to the *statistical dimension* of the allowed power spectrum of that class.
Stable Tensor Neural Networks for Rapid Deep Learning
To exemplify the elegant, matrix-mimetic algebraic structure of our $t$-NNs, we expand on recent work (Haber and Ruthotto, 2017) which interprets deep neural networks as discretizations of non-linear differential equations and introduces stable neural networks which promote superior generalization.
Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees
Qualitatively, our results are twofold: on the one hand, we show that random Fourier feature approximation can provably speed up kernel ridge regression under reasonable assumptions.
Sketching for Principal Component Regression
Principal component regression (PCR) is a useful method for regularizing linear regression.
Stochastic Chebyshev Gradient Descent for Spectral Optimization
A large class of machine learning techniques requires the solution of optimization problems involving spectral functions of parametric matrices, e. g. log-determinant and nuclear norm.
Should You Derive, Or Let the Data Drive? An Optimization Framework for Hybrid First-Principles Data-Driven Modeling
This work takes a different perspective and targets the construction of a correction model operator with implicit attributes.
Experimental Design for Non-Parametric Correction of Misspecified Dynamical Models
We consider a class of misspecified dynamical models where the governing term is only approximately known.
Faster Kernel Ridge Regression Using Sketching and Preconditioning
The preconditioner is based on random feature maps, such as random Fourier features, which have recently emerged as a powerful technique for speeding up and scaling the training of kernel-based methods, such as kernel ridge regression, by resorting to approximations.
Sharper Bounds for Regularized Data Fitting
We study regularization both in a fairly broad setting, and in the specific context of the popular and widely used technique of ridge regularization; for the latter, as applied to each of these problems, we show algorithmic resource bounds in which the {\em statistical dimension} appears in places where in previous bounds the rank would appear.
Hierarchically Compositional Kernels for Scalable Nonparametric Learning
We propose a novel class of kernels to alleviate the high computational cost of large-scale nonparametric learning with kernel methods.
Approximating the Spectral Sums of Large-scale Matrices using Chebyshev Approximations
Computation of the trace of a matrix function plays an important role in many scientific computing applications, including applications in machine learning, computational physics (e. g., lattice quantum chromodynamics), network analysis and computational biology (e. g., protein folding), just to name a few application areas.
Data Structures and Algorithms
Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels
These approximate feature maps arise as Monte Carlo approximations to integral representations of shift-invariant kernel functions (e. g., Gaussian kernel).
Subspace Embeddings for the Polynomial Kernel
Sketching is a powerful dimensionality reduction tool for accelerating statistical learning algorithms.
High-performance Kernel Machines with Implicit Distributed Optimization and Randomization
In order to fully utilize "big data", it is often required to use "big models".
Random Laplace Feature Maps for Semigroup Kernels on Histograms
With the goal of accelerating the training and testing complexity of nonlinear kernel methods, several recent papers have proposed explicit embeddings of the input data into low-dimensional feature spaces, where fast linear methods can instead be used to generate approximate solutions.
Sketching Structured Matrices for Faster Nonlinear Regression
Motivated by the desire to extend fast randomized techniques to nonlinear $l_p$ regression, we consider a class of structured regression problems.
