Simplifying UAS control and improving automation using an RBFNN for gesture and activity recognition

Abstract

A simplistic view of an unmanned aircraft is that it is an aircraft with a computer system and a radio-link that replaces the pilot. The unmanned aircraft merely forms part of the total system referred to as an Unmanned Aircraft or Aerial System (UAS). UASs have become a pervasive technology in diverse fields, from recreational to advanced military applications. Similar to many modern technologies, UASs have become less expensive and simpler to attain. A UAS can be controlled manually or autonomously through various methods. Existing UAS control methods and the possible means to simplify UAS control and improve automation techniques are discussed. The first simplified control system uses accelerometer data from a smartwatch to control the pitch, yaw, and roll of the UAS. A gesture recognition system presented in this study was implemented by using a smartwatch that captures accelerometer data, based on the hand gestures made by the pilot. The accelerometer data is then analysed by a Fast Fourier Transformation (FFT) and sent to a radial basis function neural network (RBFNN) to identify the gestures. Various automated tasks are assigned to these identified gestures to improve automation and simplify UAS control. A following function is also introduced to enable the UAS to autonomously follow the pilot, based on the GPS coordinates of the hand-held device. The UAS is further automated by using an activity recognition system that uses an accelerometer to gather data from the smartwatch while the pilot is doing an activity. The data is processed by an FFT and used to train an RBFNN model which then predicts the activity that was performed. Based on the activity that was identified, the speed of the UAS has been adjusted accordingly. The experiments show that the RBFNN can accurately distinguish between the different gestures. The RBFNN is also able to accurately predict the activity that was performed by the pilot. UAS control is simplified by enabling the UAS to autonomously follow the pilot, based on GPS coordinates, enabling the pilot to focus on other activities instead of flying the UAS. The ability of the RBFNN to identify gestures performed by the pilot enables the pilot to easily send control instructions to the UAS without the need to carry a bulky remote control in his/her hand. The RBFNN also improves UAS automation by autonomously identifying the activity the pilot is performing and then adjusting the speed accordingly, allowing the pilot to focus on other activities.