Discriminatory performance of error metrics in selected Non-linear models

Abstract

The purpose of this study is to determine the discriminatory performance of the error metrics on two non-linear models, specifically the Markov Switching Autoregressive (MS-AR) models and the Artificial Neural Networks (ANN). The inflation rate of South Africa was used as an experimental unit and quarterly data for the first quarter of 1993 to the second quarter of 2016, serving 90 observations were used. Brock, Dechert and Scheinkman (BDS) test, Cumulative Sum (CUSUM) and the Ramsey Regression Equation Specification Error Test (RESET) were employed to confirm the presence of non-linearity and instability of the data. In case of MS-AR models, the Akaike Information Criterion (AIC) was used as a best model selection criterion, while for the ANN, different learning rates and momentum values were employed for selecting the best model. The following error metrics were employed for evaluating the forecasting performance of the two competing models; Mean Absolute Error (MAE), Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPEreg), Theil’s U Test, Symmetric Mean Absolute Percentage Error (MAPEsym), Geometric Mean of Squares of Error (GMSE) and the Median Absolute Percentage Error (MdAPE). The verdict of the study was that the ANN out-performs the MS-AR models because of lesser errors produced when forecasting. With these results, the central bank can strive to achieve price stability.