Search Results for author:
Found 7 papers, 3 papers with code
Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives
We consider a gradient-based optimization method applied to a function $\mathcal{L}$ of a vector of variables $\boldsymbol{\theta}$, in the case where $\boldsymbol{\theta}$ is represented as a tuple of tensors $(\mathbf{T}_1, \cdots, \mathbf{T}_S)$.
Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review
It then explores MEMS-based applications on ultra-low power MCUs, highlighting their potential for enabling TinyML on resource-constrained devices.
Rethinking Gauss-Newton for learning over-parameterized models
This work studies the global convergence and implicit bias of Gauss Newton's (GN) when optimizing over-parameterized one-hidden layer networks in the mean-field regime.
Gaussian Pre-Activations in Neural Networks: Myth or Reality?
The study of feature propagation at initialization in neural networks lies at the root of numerous initialization designs.
An Equivalence between Bayesian Priors and Penalties in Variational Inference
We fully characterize the regularizers that can arise according to this procedure, and provide a systematic way to compute the prior corresponding to a given penalty.
Learning with Random Learning Rates
Hyperparameter tuning is a bothersome step in the training of deep learning models.
Learning with Random Learning Rates.
Hyperparameter tuning is a bothersome step in the training of deep learning mod- els.
