Quantitative analysis of near infrared spectroscopic data based on dual-band transformation and competitive adaptive reweighted sampling.

ABSTRACT

Near infrared (NIR) spectroscopy has the characteristics of rapid processing, nondestructive analysis and on-line detection. This technique has been widely used in the fields of quantitative determination and substance content analysis. However, for complex NIR spectral data, most traditional machine learning models cannot carry out effective quantitative analyses (manifested as underfitting; that is, the training effect of the model is not good). Small amounts of available data limit the performance of deep learning-based infrared spectroscopy methods, while the traditional threshold-based feature selection methods require more prior knowledge. To address the above problems, this paper proposes a competitive adaptive reweighted sampling method based on dual band transformation (DWT-CARS). DWT-CARS includes four types in total CARS based on integrated two-dimensional correlation spectrum (i2DCOS-CARS), CARS based on difference coefficient (DI-CARS), CARS based on ratio coefficient (RI-CARS) and CARS based on normalized difference coefficient (NDI-CARS). We conducted comparative experiments on three datasets; compared to traditional machine learning methods, our method achieved good results, demonstrating that this method has considerable prospects for the quantitative analysis of near-infrared spectroscopic data. To further improve the performance and stability of this method, we combined the idea of integrated modeling and constructed a partial least squares model based on Monte Carlo sampling for the samples obtained by CARS (DWT-CARS-MC-PLS). Through comparative experiments, we verified that the integrated model could further enhance the accuracy and stability of the results.