Investigation of factorisation machines and its extensions to predictive models in a statistical context

Abstract

Factorisation machines originated from the field of machine learning and have gained popularity because of the high accuracy obtained in several prediction problems, particularly in the field of recommender systems. As will be seen in the introductory discussion statisticians are largely unaware of factorisation machines and their capabilities and are seldomly considered by financial companies in their model-building endeavours. On the other hand, generalised linear models are frequently used by financial companies, and given the close relation of factorisation machines to these models, it is rather unfortunate that factorisation machines receive little attention. It is also surprising that factorisation machines are seldom if ever, mentioned as an alternative technique to benchmark models when problems in the financial services industry are discussed. Other than our papers, we have not been able to find a paper on factorisation machines published in a statistical journal. One of the reasons is probably since machine learners use terminology unfamiliar to statisticians. Almost all of the papers on factorisation machines are published in conference proceedings and in engineering and artificial intelligence journals. Although banks and insurance companies make use of open-source software, their production systems are usually coded in commercial software. Currently, commercial software is rather restricted in the routines that are offered to fit factorisation machines. So, to facilitate application of factorisation machines to a wide range of problems, a suite of fitting routines is needed. In this thesis, we introduce factorisation machines to the statistical community and show that they can be applied to a wide range of modelling problems. Specifically, we compare and relate factorisation machines to generalised linear models and we develop several fitting routines in popular commercial software. The accuracy of the routines is thoroughly checked employing simulation studies. The performance of factorisation machines is then compared to that of popular generalised linear models by using Monte Carlo simulation and several real-world data sets. Specific contributions of the thesis include the introduction of a new robust factorisation machine, based on mean absolute deviation, and logistic factorisation machines that perform well on binary classification problems where the number of events is much less than the number of non-events. We further motivate why and in what setting it will be beneficial to include factorisation machines as part of the Statistician's model-building toolkit. During our research, understanding and analysis of FMs, we concluded that FMs can potentially be used to solve a range of exciting business problems, and we include some of these as recommendations for future research. Lastly, we believe that the routines developed in this thesis will result in the widespread application of factorisation machines in business. Also, we hope that the publications resulting from this thesis will stimulate research interest on this topic by statisticians.