Search Results for author:
Found 23 papers, 5 papers with code
The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Model
Previous works have analyzed separately how forgetting is affected by either task similarity or overparameterization.
Theoretical Perspectives on Deep Learning Methods in Inverse Problems
In recent years, there have been significant advances in the use of deep learning methods in inverse problems such as denoising, compressive sensing, inpainting, and super-resolution.
Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regime
We show experimentally that in permuted MNIST image classification tasks, the generalization performance of multilayer perceptrons trained by vanilla stochastic gradient descent can be improved by overparameterization, and the extent of the performance increase achieved by overparameterization is comparable to that of state-of-the-art continual learning algorithms.
Regularized Training of Intermediate Layers for Generative Models for Inverse Problems
For both of these inversion algorithms, we introduce a new regularized GAN training algorithm and demonstrate that the learned generative model results in lower reconstruction errors across a wide range of under sampling ratios when solving compressed sensing, inpainting, and super-resolution problems.
Score-based Generative Neural Networks for Large-Scale Optimal Transport
We consider the fundamental problem of sampling the optimal transport coupling between given source and target distributions.
Generator Surgery for Compressed Sensing
We introduce a method for achieving low representation error using generators as signal priors.
Optimal Sample Complexity of Subgradient Descent for Amplitude Flow via Non-Lipschitz Matrix Concentration
We establish local convergence of subgradient descent with optimal sample complexity based on the uniform concentration of a random, discontinuous matrix-valued operator arising from the objective's gradient dynamics.
Compressive Phase Retrieval: Optimal Sample Complexity with Deep Generative Priors
Advances in compressive sensing provided reconstruction algorithms of sparse signals from linear measurements with optimal sample complexity, but natural extensions of this methodology to nonlinear inverse problems have been met with potentially fundamental sample complexity bottlenecks.
Nonasymptotic Guarantees for Spiked Matrix Recovery with Generative Priors
Many problems in statistics and machine learning require the reconstruction of a rank-one signal matrix from noisy data.
Global Convergence of Sobolev Training for Overparameterized Neural Networks
Sobolev loss is used when training a network to approximate the values and derivatives of a target function at a prescribed set of input points.
Reducing the Representation Error of GAN Image Priors Using the Deep Decoder
In this paper, we demonstrate a method for reducing the representation error of GAN priors by modeling images as the linear combination of a GAN prior with a Deep Decoder.
Removing the Representation Error of GAN Image Priors Using the Deep Decoder
In this paper, we demonstrate a method for removing the representation error of a GAN when used as a prior in inverse problems by modeling images as the linear combination of a GAN with a Deep Decoder.
Global Guarantees for Blind Demodulation with Generative Priors
That is, the objective function has a descent direction at every point outside of a small neighborhood around four hyperbolic curves.
Invertible generative models for inverse problems: mitigating representation error and dataset bias
For compressive sensing, invertible priors can yield higher accuracy than sparsity priors across almost all undersampling ratios, and due to their lack of representation error, invertible priors can yield better reconstructions than GAN priors for images that have rare features of variation within the biased training set, including out-of-distribution natural images.
Deep Denoising: Rate-Optimal Recovery of Structured Signals with a Deep Prior
Deep neural networks provide state-of-the-art performance for image denoising, where the goal is to recover a near noise-free image from a noisy image.
Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
In this paper, we propose an untrained simple image model, called the deep decoder, which is a deep neural network that can generate natural images from very few weight parameters.
Phase Retrieval Under a Generative Prior
Our formulation has provably favorable global geometry for gradient methods, as soon as $m = O(kd^2\log n)$, where $d$ is the depth of the network.
Rate-Optimal Denoising with Deep Neural Networks
Deep neural networks provide state-of-the-art performance for image denoising, where the goal is to recover a near noise-free image from a noisy observation.
Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk
We establish that in both cases, in suitable regimes of network layer sizes and a randomness assumption on the network weights, that the non-convex objective function given by empirical risk minimization does not have any spurious stationary points.
A Convex Program for Mixed Linear Regression with a Recovery Guarantee for Well-Separated Data
We introduce a convex approach for mixed linear regression over $d$ features.
ShapeFit and ShapeKick for Robust, Scalable Structure from Motion
We introduce a new method for location recovery from pair-wise directions that leverages an efficient convex program that comes with exact recovery guarantees, even in the presence of adversarial outliers.
Exact simultaneous recovery of locations and structure from known orientations and corrupted point correspondences
This recovery theorem is based on a set of deterministic conditions that we prove are sufficient for exact recovery.
ShapeFit: Exact location recovery from corrupted pairwise directions
We prove that this program recovers a set of $n$ i. i. d.
