Supervised and transfer learning for industrial equipment remaining useful life prediction under limited data

Abstract

Remaining useful life (RUL) prediction is a key component of predictive maintenance (PdM) systems, where data-driven models are used to identify degradation patterns and predict impending equipment failure. Despite the advances of Industry 4.0 and the increased availability of sensor data, limited labelled degradation data remains a challenge in PdM. However, limited research has systematically assessed how different machine learning models perform as labelled target data become increasingly scarce. Moreover, when practitioners have access to only a limited amount of labelled target data, related source-domain datasets can be used. However, the domain shift between source and target datasets can affect model performance. This study therefore evaluates classical regression, tree based, and neural-network-based models under varying data-availability constraints. Feature selection and hybrid feature extraction architectures are also investigated to determine their effect on model performance (across the varying data availability constraints). In addition, three supervised fine-tuning strategies, namely full fine-tuning, layer freezing, and progressive unfreezing, are evaluated for scenarios where source-domain data are available. For the case where no labelled target data are available, adversarial domain adaptation is implemented using a domain adversarial neural network (DANN). The experiments are conducted using the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) datasets, with FD001 used as the target dataset and FD002 and FD003 used as source-domain datasets. The results show that, within the investigated C-MAPSS datasets and experimental design, no single modelling strategy performed best across all data-availability scenarios. Neural-network-based models generally performed best when sufficient labelled target data were available. Among the evaluated neural-network-based models, the Transformer showed the greatest robustness as target training data became scarce. Under extreme training-data scarcity, the evaluated linear and tree-based models became increasingly competitive and outperformed the neural-network-based models. Furthermore, feature selection was found to mainly improve computational efficiency, particularly reducing the recurrent models' training time by approximately 87% to 97%, although its effect on predictive accuracy varied across the evaluated models. The proposed AUC-RMSE metric further showed that conclusions based solely on aggregate RMSE can differ from those obtained when prediction performance is evaluated across different stages of the equipment lifecycle. For transfer learning, the supervised fine-tuning techniques were most advantageous in the lower-data scenarios, while negative transfer was observed under some higher-data conditions. The greatest benefits were observed for the more similar of the investigated source-target domains, where all three fine-tuning strategies achieved RMSE improvements of more than 50% over the supervised target-only baseline in the data-scarce scenarios. When labelled target data were unavailable, source-only transfer remained competitive in the smaller domain-shift setting, whereas DANN provided greater benefit in the larger of the investigated domain-shift settings. In this setting, DANN improved RMSE over the source-only baseline by more than 27% across all evaluated data ranges, with the highest improvement of 38% observed under extreme scarcity. Under very low and extreme labelled target-data settings, source-only and DANN-based transfer models also became competitive with, and in some cases outperformed, the supervised GRU baseline. These findings demonstrate predictive improvements within the investigated C-MAPSS benchmark scenarios. Future work should evaluate the investigated models on additional datasets to assess the generalisability of the findings. Moreover, the evaluation on real world industrial degradation data could further assess their practical applicability.