Search Results for author:
Found 10 papers, 3 papers with code
Gradient is All You Need?
The fundamental value of such link between CBO and SGD lies in the fact that CBO is provably globally convergent to global minimizers for ample classes of nonsmooth and nonconvex objective functions, hence, on the one side, offering a novel explanation for the success of stochastic relaxations of gradient descent.
Finite Sample Identification of Wide Shallow Neural Networks with Biases
Artificial neural networks are functions depending on a finite number of parameters typically encoded as weights and biases.
Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders
Autoencoding is a popular method in representation learning.
Landscape analysis of an improved power method for tensor decomposition
In this work, we consider the optimization formulation for symmetric tensor decomposition recently introduced in the Subspace Power Method (SPM) of Kileel and Pereira.
Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples
In this paper we approach the problem of unique and stable identifiability of generic deep artificial neural networks with pyramidal shape and smooth activation functions from a finite number of input-output samples.
A deep network construction that adapts to intrinsic dimensionality beyond the domain
We study the approximation of two-layer compositions $f(x) = g(\phi(x))$ via deep networks with ReLU activation, where $\phi$ is a geometrically intuitive, dimensionality reducing feature map.
Estimating covariance and precision matrices along subspaces
We also show that estimation of precision matrices is almost independent of the condition number of the covariance matrix.
Robust and Resource Efficient Identification of Two Hidden Layer Neural Networks
Gathering several approximate Hessians allows reliably to approximate the matrix subspace $\mathcal W$ spanned by symmetric tensors $a_1 \otimes a_1 ,\dots, a_{m_0}\otimes a_{m_0}$ formed by weights of the first layer together with the entangled symmetric tensors $v_1 \otimes v_1 ,\dots, v_{m_1}\otimes v_{m_1}$, formed by suitable combinations of the weights of the first and second layer as $v_\ell=A G_0 b_\ell/\|A G_0 b_\ell\|_2$, $\ell \in [m_1]$, for a diagonal matrix $G_0$ depending on the activation functions of the first layer.
Nonlinear generalization of the monotone single index model
This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses.
Adaptive multi-penalty regularization based on a generalized Lasso path
Another advantage of our algorithm is that it provides an overview on the solution stability over the whole range of parameters.
