Search Results for author:
Found 18 papers, 5 papers with code
On the Nystrom Approximation for Preconditioning in Kernel Machines
Scalable algorithms for learning kernel models need to be iterative in nature, but convergence can be slow due to poor conditioning.
Mechanism of feature learning in convolutional neural networks
We then demonstrate the generality of our result by using the patch-based AGOP to enable deep feature learning in convolutional kernel machines.
Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks
We study the local dynamics of GDA for training a GAN with a kernel-based discriminator.
Toward Large Kernel Models
Recent studies indicate that kernel machines can often perform similarly or better than deep neural networks (DNNs) on small datasets.
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features
In recent years neural networks have achieved impressive results on many technological and scientific tasks.
Instability and Local Minima in GAN Training with Kernel Discriminators
Generative Adversarial Networks (GANs) are a widely-used tool for generative modeling of complex data.
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
In this work we argue that while benign overfitting has been instructive and fruitful to study, many real interpolating methods like neural networks do not fit benignly: modest noise in the training set causes nonzero (but non-infinite) excess risk at test time, implying these models are neither benign nor catastrophic but rather fall in an intermediate regime.
A note on Linear Bottleneck networks and their Transition to Multilinearity
In this work we show that linear networks with a bottleneck layer learn bilinear functions of the weights, in a ball of radius $O(1)$ around initialization.
On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions
``Benign overfitting'', the ability of certain algorithms to interpolate noisy training data and yet perform well out-of-sample, has been a topic of considerable recent interest.
Kernel Methods and Multi-layer Perceptrons Learn Linear Models in High Dimensions
Empirical observation of high dimensional phenomena, such as the double descent behaviour, has attracted a lot of interest in understanding classical techniques such as kernel methods, and their implications to explain generalization properties of neural networks.
Implicit Bias of Linear RNNs
The degree of this bias depends on the variance of the transition kernel matrix at initialization and is related to the classic exploding and vanishing gradients problem.
Matrix Inference and Estimation in Multi-Layer Models
In the two-layer neural-network learning problem, this scaling corresponds to the case where the number of input features, as well as training samples, grow to infinity but the number of hidden nodes stays fixed.
Low-Rank Nonlinear Decoding of $μ$-ECoG from the Primary Auditory Cortex
This decoding problem is particularly challenging due to the complexity of neural responses in the auditory cortex and the presence of confounding signals in awake animals.
Generalization Error of Generalized Linear Models in High Dimensions
We provide a general framework to characterize the asymptotic generalization error for single-layer neural networks (i. e., generalized linear models) with arbitrary non-linearities, making it applicable to regression as well as classification problems.
Inference in Multi-Layer Networks with Matrix-Valued Unknowns
We consider the problem of inferring the input and hidden variables of a stochastic multi-layer neural network from an observation of the output.
Inference with Deep Generative Priors in High Dimensions
This paper presents a novel algorithm, Multi-Layer Vector Approximate Message Passing (ML-VAMP), for inference in multi-layer stochastic neural networks.
High-Dimensional Bernoulli Autoregressive Process with Long-Range Dependence
We derive precise upper bounds on the mean-squared estimation error in terms of the number of samples, dimensions of the process, the lag $p$ and other key statistical properties of the model.
Asymptotics of MAP Inference in Deep Networks
Deep generative priors are a powerful tool for reconstruction problems with complex data such as images and text.
