Search Results for author:
Found 25 papers, 5 papers with code
Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks
We study the local dynamics of GDA for training a GAN with a kernel-based discriminator.
Instability and Local Minima in GAN Training with Kernel Discriminators
Generative Adversarial Networks (GANs) are a widely-used tool for generative modeling of complex data.
Kernel Methods and Multi-layer Perceptrons Learn Linear Models in High Dimensions
Empirical observation of high dimensional phenomena, such as the double descent behaviour, has attracted a lot of interest in understanding classical techniques such as kernel methods, and their implications to explain generalization properties of neural networks.
Asymptotics of Ridge Regression in Convolutional Models
We show the double descent phenomenon in our experiments for convolutional models and show that our theoretical results match the experiments.
Implicit Bias of Linear RNNs
The degree of this bias depends on the variance of the transition kernel matrix at initialization and is related to the classic exploding and vanishing gradients problem.
Matrix Inference and Estimation in Multi-Layer Models
In the two-layer neural-network learning problem, this scaling corresponds to the case where the number of input features, as well as training samples, grow to infinity but the number of hidden nodes stays fixed.
Low-Rank Nonlinear Decoding of $μ$-ECoG from the Primary Auditory Cortex
This decoding problem is particularly challenging due to the complexity of neural responses in the auditory cortex and the presence of confounding signals in awake animals.
Generalization Error of Generalized Linear Models in High Dimensions
We provide a general framework to characterize the asymptotic generalization error for single-layer neural networks (i. e., generalized linear models) with arbitrary non-linearities, making it applicable to regression as well as classification problems.
Inference in Multi-Layer Networks with Matrix-Valued Unknowns
We consider the problem of inferring the input and hidden variables of a stochastic multi-layer neural network from an observation of the output.
Input-Output Equivalence of Unitary and Contractive RNNs
Unitary recurrent neural networks (URNNs) have been proposed as a method to overcome the vanishing and exploding gradient problem in modeling data with long-term dependencies.
Inference with Deep Generative Priors in High Dimensions
This paper presents a novel algorithm, Multi-Layer Vector Approximate Message Passing (ML-VAMP), for inference in multi-layer stochastic neural networks.
High-Dimensional Bernoulli Autoregressive Process with Long-Range Dependence
We derive precise upper bounds on the mean-squared estimation error in terms of the number of samples, dimensions of the process, the lag $p$ and other key statistical properties of the model.
Asymptotics of MAP Inference in Deep Networks
Deep generative priors are a powerful tool for reconstruction problems with complex data such as images and text.
Plug-in Estimation in High-Dimensional Linear Inverse Problems: A Rigorous Analysis
Estimating a vector $\mathbf{x}$ from noisy linear measurements $\mathbf{Ax}+\mathbf{w}$ often requires use of prior knowledge or structural constraints on $\mathbf{x}$ for accurate reconstruction.
Information Theory Information Theory
Inference in Deep Networks in High Dimensions
In inverse problems that use these networks as generative priors on data, one must often perform inference of the inputs of the networks from the outputs.
Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems
We show that the parameter estimates and mean squared error (MSE) of x in each iteration converge to deterministic limits that can be precisely predicted by a simple set of state evolution (SE) equations.
Vector Approximate Message Passing
The approximate message passing (AMP) algorithm recently proposed by Donoho, Maleki, and Montanari is a computationally efficient iterative approach to SLR that has a remarkable property: for large i. i. d.\ sub-Gaussian matrices $\mathbf{A}$, its per-iteration behavior is rigorously characterized by a scalar state-evolution whose fixed points, when unique, are Bayes optimal.
Information Theory Information Theory
Learning and Free Energies for Vector Approximate Message Passing
Like the AMP proposed by Donoho, Maleki, and Montanari in 2009, VAMP is characterized by a rigorous state evolution (SE) that holds under certain large random matrices and that matches the replica prediction of optimality.
Expectation Consistent Approximate Inference: Generalizations and Convergence
Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems.
Scalable Inference for Neuronal Connectivity from Calcium Imaging
In this work, we propose a computationally fast method for the state estimation based on a hybrid of loopy belief propagation and approximate message passing (AMP).
Approximate Message Passing with Consistent Parameter Estimation and Applications to Sparse Learning
We present a method, called adaptive generalized approximate message passing (Adaptive GAMP), that enables joint learning of the statistics of the prior and measurement channel along with estimation of the unknown vector $\xbf$.
Neural Reconstruction with Approximate Message Passing (NeuRAMP)
Many functional descriptions of spiking neurons assume a cascade structure where inputs are passed through an initial linear filtering stage that produces a low-dimensional signal that drives subsequent nonlinear stages.
Orthogonal Matching Pursuit From Noisy Random Measurements: A New Analysis
Orthogonal matching pursuit (OMP) is a widely used greedy algorithm for recovering sparse vectors from linear measurements.
Asymptotic Analysis of MAP Estimation via the Replica Method and Compressed Sensing
It is shown that with large random linear measurements and Gaussian noise, the asymptotic behavior of the MAP estimate of an n-dimensional vector ``decouples as n scalar MAP estimators.
Resolution Limits of Sparse Coding in High Dimensions
Recent research suggests that neural systems employ sparse coding.
