Regularization for convolutional kernel tensors to avoid unstable gradient problem in convolutional neural networks

no code implementations • 5 Feb 2021 • Pei-Chang Guo

In order to help avoid the exploding/vanishing gradient problem, it is desirable that the singular values of each transformation matrix are not large/small in the training process.

A Frobenius norm regularization method for convolutional kernels to avoid unstable gradient problem

no code implementations • 25 Jul 2019 • Pei-Chang Guo

We propose a new penalty function for a convolutional kernel to let the singular values of the corresponding transformation matrix are bounded around $1$.

On regularization for a convolutional kernel in neural networks

no code implementations • 12 Jun 2019 • Pei-Chang Guo, Qiang Ye

Convolutional neural network is an important model in deep learning.