Search Results for author:
Found 10 papers, 5 papers with code
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
Adan first reformulates the vanilla Nesterov acceleration to develop a new Nesterov momentum estimation (NME) method, which avoids the extra overhead of computing gradient at the extrapolation point.
Global Convergence of Over-parameterized Deep Equilibrium Models
A deep equilibrium model (DEQ) is implicitly defined through an equilibrium point of an infinite-depth weight-tied model with an input-injection.
High Quality Segmentation for Ultra High-resolution Images
To segment 4K or 6K ultra high-resolution images needs extra computation consideration in image segmentation.
Optimization inspired Multi-Branch Equilibrium Models
Works have shown the strong connections between some implicit models and optimization problems.
Optimization Induced Equilibrium Networks
In this paper, we investigate an emerging question: can an implicit equilibrium model's equilibrium point be regarded as the solution of an optimization problem?
AC-VAE: Learning Semantic Representation with VAE for Adaptive Clustering
Experimental evaluations show that the proposed method outperforms state-of-the-art representation learning methods in terms of neighbor clustering accuracy.
Fast and Differentiable Matrix Inverse and Its Extension to SVD
First of all, to perform matrix inverse, we provide a differentiable yet efficient way, named LD-Minv, which is a learnable deep neural network (DNN) with each layer being an $L$-th order matrix polynomial.
Maximum-and-Concatenation Networks
While successful in many fields, deep neural networks (DNNs) still suffer from some open problems such as bad local minima and unsatisfactory generalization performance.
Differentiable Linearized ADMM
Recently, a number of learning-based optimization methods that combine data-driven architectures with the classical optimization algorithms have been proposed and explored, showing superior empirical performance in solving various ill-posed inverse problems, but there is still a scarcity of rigorous analysis about the convergence behaviors of learning-based optimization.
Matrix Recovery with Implicitly Low-Rank Data
To tackle this issue, we propose a novel method for matrix recovery in this paper, which could well handle the case where the target matrix is low-rank in an implicit feature space but high-rank or even full-rank in its original form.
