Hyper-parameters Tuning of Artificial Neural Networks: An Application in the Field of Recommender Systems

Abstract

In this work, we carry out the hyper-parameters tuning of a Machine Learning (ML) Recommender Systems (RS) which utilizes an Artificial Neural Network (ANN), called CATA++. We have performed tuning of the activation function, weight initialization and training epochs of CATA++ in order to improve both training and performance. During the experiments, a variety of state-of-the-art activation functions have been tested: ReLU, LeakyReLU, ELU, SineReLU, GELU, Mish, Swish and Flatten-T Swish. Additionally, various weight initializers have been tested, such as: XavierGlorot, Orthogonal, He, Lecun. Moreover, we ran experiments with different epochs number from 10 to 150. We have used data from CiteULike and AMiner Citation Network. The recorded metrics (Recall, nDCG) indicate that hyper-parameters tuning can reduce notably the necessary training time, while the recommendation performance is significantly improved (up to +44.2% Recall). © 2022, Springer Nature Switzerland AG.