Search Results for author:
Found 9 papers, 0 papers with code
On the optimization and generalization of overparameterized implicit neural networks
We show that global convergence is guaranteed, even if only the implicit layer is trained.
Gradient Descent Optimizes Infinite-Depth ReLU Implicit Networks with Linear Widths
Implicit deep learning has recently become popular in the machine learning community since these implicit models can achieve competitive performance with state-of-the-art deep networks while using significantly less memory and computational resources.
A global convergence theory for deep ReLU implicit networks via over-parameterization
Implicit deep learning has received increasing attention recently due to the fact that it generalizes the recursive prediction rules of many commonly used neural network architectures.
Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems
Our method leverages structured sparsification to reduce computational cost without hurting the model capacity at the end of offline training so that a full-size model is available in the recurring training stage to learn new data in real-time.
Randomized Bregman Coordinate Descent Methods for Non-Lipschitz Optimization
Further, we show that the iteration complexity of the proposed method is $O(n\varepsilon^{-2})$ to achieve $\epsilon$-stationary point, where $n$ is the number of blocks of coordinates.
Leveraging Two Reference Functions in Block Bregman Proximal Gradient Descent for Non-convex and Non-Lipschitz Problems
In the applications of signal processing and data analytics, there is a wide class of non-convex problems whose objective function is freed from the common global Lipschitz continuous gradient assumption (e. g., the nonnegative matrix factorization (NMF) problem).
A Forest from the Trees: Generation through Neighborhoods
In this work, we propose to learn a generative model using both learned features (through a latent space) and memories (through neighbors).
DID: Distributed Incremental Block Coordinate Descent for Nonnegative Matrix Factorization
We propose a novel distributed algorithm, called \textit{distributed incremental block coordinate descent} (DID), to solve the problem.
Degrees of Freedom in Deep Neural Networks
The degrees of freedom in deep networks are dramatically smaller than the number of parameters, in some real datasets several orders of magnitude.
