Investigating low-cost semi-automatic Digital Twin generation for South African deep-level mines

Abstract

The South African deep-level mining industry is shifting towards the Industry 4.0 paradigm,which is the digitisation of complex industrial systems through the implementation and integration of advanced technologies. This paradigm comprises predictive maintenance, real-time remote monitoring and control, and process optimisation, among others. However, financial constraints slow the adoption of the paradigm in the mining industry, restricting benefits. One of the tools enabled by the Industry 4.0 paradigm is Digital Twins (DTs). However, DTs are time-consuming to create manually and existing semi-automatic generation methods, such as Light Detection and Ranging (LiDAR), are typically expensive and require specialised equipment. In contrast, studies from other industries indicate that low-cost localisation and mapping techniques can be utilised to enable the e!cient creation of DTs. Therefore, there is a need for a low-cost semi-automatic DT generation method to increase the adoption of Industry 4.0 in the South African deep-level mining industry. By investigating the e"ectiveness of low-cost localisation and mapping techniques, the financially constrained mining sector can take a step closer towards feasible, fully automatic DT generation. This study proposes a method for low-cost semi-automatic DT generation consisting of three parts. Firstly, a localisation method for o#ine environments that utilises an inertial measurement unit typically found in mobile devices. Secondly, a method to extract information automatically from images captured inside deep-level mines using image segmentation, trained by utilising a synthetic dataset generator. Lastly, the integration and extension of the previous two parts to create the low-cost semi-automatic DT generation method. Each method was independently verified to ensure the accuracy of each underlying step and the overall proposed method. The proposed Kalman Filter-based inertial navigation method performed well in its verification. An error of only 2.4 m was obtained for a total walking distance of 80.6 m, with peak signal-to-noise ratio (PSNR) values of 12.84 dB, 35.14 dB, and 19.19 dB for the accelerometer, gyroscope, and magnetometer data. The verification of the proposed deep-learning image segmentation method, which includes a synthetic dataset generator, also produced promising results. Unseen real-world images were segmented with an Intersection over Union (IoU) score of 34.15 and a Panoptic Quality (PQ) score of 32.41. A recent survey study found an average IoU of 39.3 and an average PQ of 40.31 from the included methods trained using a real-world dataset. The slightly lower performance achieved in this study can be attributed to the reality gap of using a synthetic dataset. The verification of the overall proposed method showed PSNR results of 12.30 dB, 36.16 dB, and 17.99 dB for the path estimation. The captured images also showed successful segmentation of objects with high confidence scores of classes from the synthetic dataset created. The captured images also showed successful segmentation of objects with high confidence scores of classes from the created synthetic dataset. The validation of the overall proposed method was done using a case study South African deep-level mine. The results obtained from the five sample sets showed successful path estimation, with PSNR results of 15.31 dB, 39.75 dB, and 16.86 dB, similar to the verification. The image segmentation also produced high-performing results, even in low-light conditions encountered in the deep-level mine. Further improvements can be made to increase the robustness of each of the proposed methods. However, this research shows that the proposed modular SLAM approach is a feasible method for low-cost semi-automatic DT generation, and it creates a stepping stone towardsfully automatic low-cost DT generation.