Deep Neural Net with Dynamic Neuron Addition
This is an approach of designing an algorithm that adds neurons dynamically upon confirmation of meeting a certain threshold of samples as input to a deep neural network. The idea lends strongly from the concept of neurogenesis, in that the neural network capacity grows, as inputs are fed to it. The formula can be written as Z = X * W + b, Y = activation(Z), where Z is the output of the dense layers before being fed to the activation layer. X as the input matrix with dimensions(m,n), where m is the number of samples (batch size), and n is the number of input features, W as the weight matrix for the layer with dimensions (n,k), where k is the number of neurons in the layer. B as the bias matrix for the layer with dimensions (1,k). Z as the weighted sum of inputs, which is the output of the layer before applying the activation function. It has dimensions (m,k). Y is the output matrix after applying the activation function. It also has dimensions (m,k). It uses a simple feedforward architecture with dynamic neuron addition to adapt to changing data patterns. The NN was tested using the spiral_data dataset, and achieves 75% performance accuracy using a performance metric known as cross-validation.
PDF