Search Results for author:
Found 9 papers, 2 papers with code
Quaternion recurrent neural network with real-time recurrent learning and maximum correntropy criterion
We develop a robust quaternion recurrent neural network (QRNN) for real-time processing of 3D and 4D data with outliers.
The HR-Calculus: Enabling Information Processing with Quaternion Algebra
From their inception, quaternions and their division algebra have proven to be advantageous in modelling rotation/orientation in three-dimensional spaces and have seen use from the initial formulation of electromagnetic filed theory through to forming the basis of quantum filed theory.
Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case
Adam is a commonly used stochastic optimization algorithm in machine learning.
Convex Quaternion Optimization for Signal Processing: Theory and Applications
Convex optimization methods have been extensively used in the fields of communications and signal processing.
UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization
Adam-type algorithms have become a preferred choice for optimisation in the deep learning setting, however, despite success, their convergence is still not well understood.
Last-iterate convergence analysis of stochastic momentum methods for neural networks
The stochastic momentum method is a commonly used acceleration technique for solving large-scale stochastic optimization problems in artificial neural networks.
Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descent
The momentum stochastic gradient descent uses the accumulated gradient as the updated direction of the current parameters, which has a faster training speed.
A decreasing scaling transition scheme from Adam to SGD
Adaptive gradient algorithm (AdaGrad) and its variants, such as RMSProp, Adam, AMSGrad, etc, have been widely used in deep learning.
Quaternion Gradient and Hessian
The optimization of real scalar functions of quaternion variables, such as the mean square error or array output power, underpins many practical applications.
