Affiliations:
Department of Systems, Universidad Autónoma Metropolitana Unidad Azcapotzalco, México
Correspondence:
[*]
Corresponding author. Oscar Herrera, Department of Systems, Universidad Autónoma Metropolitana Unidad Azcapotzalco, México. E-mail: oha@azc.uam.mx.
Abstract: Traditionally, a few activation functions have been considered in neural networks, including bounded functions such as threshold, sigmoidal and hyperbolic-tangent, as well as unbounded ReLU, GELU, and Soft-plus, among other functions for deep learning, but the search for new activation functions still being an open research area. In this paper, wavelets are reconsidered as activation functions in neural networks and the performance of Gaussian family wavelets (first, second and third derivatives) are studied together with other functions available in Keras-Tensorflow. Experimental results show how the combination of these activation functions can improve the performance and supports the idea of extending the list of activation functions to wavelets which can be available in high performance platforms.
Keywords: deep learning, neural network, activation functions, wavelets, Keras-Tensorflow
