Loss Surface Modality of Feed-Forward Neural Network Architectures

no code implementations • 24 May 2019 • Anna Sergeevna Bosman, Andries Engelbrecht, Mardé Helbig

An increase in the hidden layer width is shown to effectively reduce the number of local minima, and simplify the shape of the global attractor.

Visualising Basins of Attraction for the Cross-Entropy and the Squared Error Neural Network Loss Functions

no code implementations • 8 Jan 2019 • Anna Sergeevna Bosman, Andries Engelbrecht, Mardé Helbig

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large.