Development of soil spectroscopy calibration and prediction models for precision agriculture within South Africa

Abstract

This thesis aimed to enhance the accuracy and applicability of soil spectroscopy prediction models for South Africa by addressing the limitations of global soil spectral libraries, for example the Open Soil Spectral Library (OSSL), which often underrepresent South African soils. The research focused on improving soil property predictions by integrating local data, employing advanced machine learning techniques, and optimising the calibration and validation of models for the unique soil conditions of the Western Highveld region. The study utilised a dataset of 772 soil samples from the Western Highveld region, encompassing fundamental agricultural soil properties pH (KCl), P, and exchangeable cations (Ca, Mg, K, Na). Mid-infrared (MIR) and nearinfrared (NIR) spectral data were collected and combined to form a comprehensive dataset. Advanced pre-processing techniques were applied to improve spectral data quality, including noise reduction, scatter correction, and feature enhancement. The dataset was then split into training and validation subsets using a conditioned Latin hypercube sampling approach to ensure robust model development. Machine learning techniques were central to the methodology. Cubist regression and convolutional neural networks (CNN) were employed to develop predictive models for soil properties. CNN demonstrated superior performance in capturing complex spectral relationships compared to traditional regression methods. The thesis also investigated the "spiking" process, a targeted integration of local spectral data into the global OSSL library, to address the underrepresentation of South African soils. This approach was evaluated at various spiking levels to determine the optimal balance between local and global data. The core findings highlight the effectiveness of combining MIR and NIR spectral data for improving soil property predictions. Combined spectral models outperformed single-spectrum models, significantly reducing prediction error for properties pH (KCl) and exchangeable Ca. Tailored pre-processing techniques provided additional improvements for specific soil properties, though raw combined spectral data often performed comparably. Machine learning models, particularly CNN, showed high predictive accuracy and robustness, outperforming traditional methods in handling the complexity and variability of soil spectral data. The spiking method demonstrated that integrating local data into global spectral libraries improves prediction accuracy for underrepresented regions. Models incorporating local data achieved superior performance compared to those relying solely on global data. However, excessive spiking introduced overfitting and reduced model generalisation, underscoring the importance of balancing local and global data contributions. This thesis contributes to soil spectroscopy by developing a scalable framework for integrating local soil data into global spectral libraries and demonstrating the value of advanced machine-learning techniques in predictive modelling. By addressing the limitations of current models, the research supports establishing a South African Soil Spectral Library, enabling more accurate soil property assessments for agricultural and environmental applications. These advancements pave the way for more effective land management and sustainable farming practices in South Africa and other underrepresented regions.