A methodology for the global optimization of this family of activation functions with free parameter and the weights of the connections between the processing units of the neural network is used.
This works proposes a methodology to searching for automatically Artificial Neural Networks (ANN) by using Cellular Genetic Algorithm (CGA).
In the current work, we performed a case study using meta-learning to choose the number of hidden nodes for MLP networks, which is an important parameter to be defined aiming a good networks performance.
The entropic out-of-distribution detection solution uses the IsoMax loss for training and the entropic score for out-of-distribution detection.
We propose a Similarity-Based Stratified Splitting (SBSS) technique, which uses both the output and input space information to split the data.
In this paper, we argue that the unsatisfactory out-of-distribution (OOD) detection performance of neural networks is mainly due to the SoftMax loss anisotropy and propensity to produce low entropy probability distributions in disagreement with the principle of maximum entropy.
Consequently, we propose IsoMax, a loss that is isotropic (distance-based) and produces high entropy (low confidence) posterior probability distributions despite still relying on cross-entropy minimization.
Besides, statistical significant performance assessments (p<0. 05) showed DReLU enhanced the test accuracy presented by ReLU in all scenarios.