Automatic modulation classification: A study in robustness against adversarial attacks

Abstract

Automatic modulation classification (AMC) is the systematic identification of the modulation scheme used by signals without any prior knowledge of the signal. The AMC process can be rather difficult, depending on the signal channel which a transmitted signal has been subjected to. Historically, AMC has been approached using a feature-based, likelihood based, or clustering approach, which we consider as non-deep learning (NDL) methods. In more recent research, the focus has shifted to deep learning (DL) methods using deep neural networks (DNNs) for AMC, as they can outperform older NDL methods in more complex signal channels. However, this good performance from the DNNs comes with the vulnerability to adversarial attacks. Adversarial attacks are subtle perturbations added to the input signals, which are visually imperceptible, but cause the DNN to misclassify the modulation type of the input signal. White-box adversarial attacks have complete knowledge of the DNN's architecture and parameters, whereas with black-box attacks this information is not available. In this study, we investigate the robustness of different AMC methods against white-box adversarial attacks. Our selected NDL methods consist of the quasi-hybrid likelihood ratio test (QHLRT), a clustering classifier, and a k-nearest neighbour (KNN) using higherorder cumulants. These approaches are selected to cover the different NDL approaches to AMC. We compare these NDL methods against each other and against a DL method, namely the parameter estimation and transformation-based CNN-GRU deep neural network (PET-CGDNN), in the presence and absence of adversarial attacks, using a dataset that simulates an additive white Gaussian noise (AWGN) signal channel. The white-box adversarial attacks consists of the fast gradient method (FGM), the projected gradient descent (PGD), and the DeepFool attacks. We use two different approaches to our adversarial attacks, that is a fixed-size attack and a varying-size attack that scales with a perturbation to-noise ratio (PNR). We generate adversarial attacks using a trained PET-CGDNN model. The adversarial attacks cause a significant deterioration in the classification performance of the DNN. With the FGM and PGD attacks, we find limited transferability from the attacks to the NDL methods, as they do not result in significant changes in classification performance. Against the DeepFool attacks, the classification performance of all our classifiers is significantly deteriorated. Furthermore, we propose the design of a hybrid classifier that combines the DL and NDL techniques, through a support vector machine (SVM) which acts as a switching mechanism, deciding whether to use the DNN or the NDL method on an input. Our hybrid classifier comprises the PET-CGDNN and the KNN using higher-order cumulants. The hybrid classifier shows comparable performance to the DNN when classifying unperturbed signals. The hybrid classifier shows a clear improvement in robustness against FGM attacks, but only a modest improvement against PGD attacks. Only a marginal improvement in robustness is observed against DeepFool attacks.