Evaluating feature attribution for multivariate time series data

Abstract

Understanding why a Deep Neural Network (DNN) produces a specific prediction can be as important as making an accurate prediction. The development of such interpretability strategies for DNNs is an active field of research. The current study aims to develop a testbed that can be used to examine the performance of deep learning interpretability techniques for multivariate time series data. Apart from some transformer-based techniques, most existing interpretability techniques were not specifically developed for time series data, but were rather adapted from other contexts, such as computer vision. In this study, we create a synthetic testbed that can be used to evaluate existing interpretability techniques specifically for time series data. The testbed provides a controlled environment within which datasets with known and tunable characteristics can be created. The purpose of the testbed is to evaluate the performance and accuracy of interpretability techniques, to allow for a fair comparison of their strengths and weaknesses. We approached this task by first identifying the characteristics of time series data that were important to control and then develop a system capable of generating the required data. A quantitative metric was then developed to compare existing techniques. This metric's effectiveness is demonstrated by applying it to synthetic test datasets. Each dataset represents known characteristics that can be used as ground truth during the evaluation of interpretability techniques. The key concepts explored in this research include time series data, existing interpretability techniques, temporal convolutional networks, and time-shifted multilayer perceptrons. The research results aim to inform the development of interpretability techniques specifically designed for time series data and advance the understanding of the challenges and opportunities in this field