Comparison of 2D and 3D convolutional neural networks in hyperspectral image analysis of fruits applied to orange bruise detection.

Recent advances in hyperspectral imaging (HSI) have demonstrated its ability to detect defects in fruit that may not be visible in RGB images. HSIs can be considered 3D images containing two spatial dimensions and one spectral dimension. Therefore, the first question that arises is how to process this type of information, either using 2D or 3D models. In this study, HSI in the 550-900 nm spectral range was used to detect bruising in oranges. Sixty samples of Thompson oranges were subjected to a mechanical bruising process, and HSIs were taken at different time intervals: before bruising, and 8 and 16 h after bruising. The samples were then classified using two convolutional neural network (CNN) models, a shallow 7-layer network (CNN-7) and a deep 18-layer network (CNN-18). In addition, two different input processing approaches are used: using 2D information from each band, and using the full 3D data from each HSI. The 3D models were the most accurate, with 94% correct classification for 3D-CNN-18, compared to 90% for 3D-CNN-7, and less than 83% for the 2D models. Our study suggests that 3D HSI may be a more effective technique for detecting fruit bruising, allowing the development of a fast, accurate, and nondestructive method for fruit sorting. PRACTICAL APPLICATION: Orange bruises can reduce the market value of food, which is why the food processing industry needs to carry out quality inspections. An effective way to perform this inspection is by using hyperspectral images that can be processed with 2D or 3D models, either with deep or shallow neural networks. The results of the comparison performed in this work can be useful for the development of more accurate and efficient bruise detection methods for fruit inspection.

Intelligent detection of citrus fruit pests using machine vision system and convolutional neural network through transfer learning technique.

Plant pests and diseases play a significant role in reducing the quality of agricultural products. As one of the most important plant pathogens, pests like Mediterranean fruit fly cause significant damage to crops and thus annually farmers face a lot of loss in their products. Therefore, the use of modern and non-destructive methods such as machine vision systems and deep learning for early detection of pests in agricultural products is of particular importance. In this study, citrus fruit images were taken in three stages: 1) before pest infestation, 2) beginning of fruit infestation, and 3) eight days after the second stage, in natural light conditions (7000-11,000 lux). A total of 1519 images were prepared for all classes. To classify the images, 70% of the images were used for the network training stage, 10% and 20% of the images were used for the validation and testing stages. Four pre-trained CNN models, namely ResNet-50, GoogleNet, VGG-16 and AlexNet as well as the SGDm, RMSProp and Adam optimization algorithms were used to identify and classify healthy fruit and fruit infected with the Mediterranean fly. The results of evaluating the models in the pest outbreak stage showed that the VGG-16 model with the help of SGDm algorithm had the best efficiency with the highest detection accuracy and F1 of 98.33% and 98.36%, respectively. The evaluation of the third stage showed that the AlexNet model with the help of SGDm algorithm had the best result with the highest detection accuracy and F1 of 99.33% and 99.34%, respectively. AlexNet model using SGDm optimization algorithm had the shortest network training time (323 s). The results of this study showed that convolutional neural network method and machine vision system can be effective in controlling and managing pests in orchards and other agricultural products.

Inner Properties Estimation of Gala Apple Using Spectral Data and Two Statistical and Artificial Intelligence Based Methods.

Fruits provide various vitamins to the human body. The chemical properties of fruits provide useful information to researchers, including determining the ripening time of fruits and the lack of nutrients in them. Conventional methods for determining the chemical properties of fruits are destructive and time-consuming methods that have no application for online operations. For that, various researchers have conducted various studies on non-destructive methods, which are currently in the research and development stage. Thus, the present paper focusses on a non-destructive method based on spectral data in the 200-1100-nm region for estimation of total soluble solids and BrimA in Gala apples. The work steps included: (1) collecting different samples of Gala apples at different stages of maturity; (2) extracting spectral data of samples and pre-preprocessing them; (3) measuring the chemical properties of TSS and BrimA; (4) selecting optimal (effective) wavelengths using artificial neural network-simulated annealing algorithm (ANN-SA); and (5) estimating chemical properties based on partial least squares regression (PLSR) and hybrid artificial neural network known as the imperialist competitive algorithm (ANN-ICA). It should be noted that, in order to investigate the validity of the methods, the estimation algorithm was repeated 500 times. In the end, the results displayed that, in the best training, the ANN-ICA predicted the TSS and BrimA with correlation coefficients of 0.963 and 0.965 and root mean squared error of 0.167% and 0.596%, respectively.

Nondestructive nitrogen content estimation in tomato plant leaves by Vis-NIR hyperspectral imaging and regression data models.

The present study aims to estimate nitrogen (N) content in tomato (Solanum lycopersicum L.) plant leaves using optimal hyperspectral imaging data by means of computational intelligence [artificial neural networks and the differential evolution algorithm (ANN-DE), partial least squares regression (PLSR), and convolutional neural network (CNN) regression] to detect potential plant stress to nutrients at early stages. First, pots containing control and treated tomato plants were prepared; three treatments (categories or classes) consisted in the application of an overdose of 30%, 60%, and 90% nitrogen fertilizer, called N-30%, N-60%, N-90%, respectively. Tomato plant leaves were then randomly picked up before and after the application of nitrogen excess and imaged. Leaf images were captured by a hyperspectral camera, and nitrogen content was measured by laboratory ordinary destructive methods. Two approaches were studied: either using all the spectral data in the visible (Vis) and near infrared (NIR) spectral bands, or selecting only the three most effective wavelengths by an optimization algorithm. Regression coefficients (R) were 0.864±0.027 for ANN-DE, 0.837±0.027 for PLSR, and 0.875±0.026 for CNN in the first approach, over the test set. The second approach used different models for each treatment, achieving R values for all the regression methods above 0.96; however, it needs a previous classification stage of the samples in one of the three nitrogen excess classes under consideration.

Nondestructive estimation of three apple fruit properties at various ripening levels with optimal Vis-NIR spectral wavelength regression data.

Nondestructive estimation of fruit properties during their ripening stages ensures the best value for producers and vendors. Among common quality measurement methods, spectroscopy is popular and enables physicochemical properties to be nondestructively estimated. The current study aims to nondestructively predict tissue firmness (kgf/cm), acidity (pH level) and starch content index (%) in apples (Malus M. pumila) samples (Fuji var.) at various ripening stages using visible/near infrared (Vis-NIR) spectral data in 400-1000 nm wavelength range. Results show that non-linear regression done by an artificial neural network-cultural algorithm (ANN-CA) was able to properly estimate the investigated fruit properties. Moreover, the performance of the proposed method was evaluated for Vis-NIR data based on optimal NIR wavelength values selected by a genetic optimization tool. Regression coefficients ( R ) in estimated acidity, tissue firmness, and starch content properties were R = 0.930 ± 0.014 , R = 0.851 ± 0.014 , and R = 0.974 ± 0.006 , respectively, using only the three most effective wavelengths from the acquired spectra.

Classification of Cucumber Leaves Based on Nitrogen Content Using the Hyperspectral Imaging Technique and Majority Voting.

Improper usage of nitrogen in cucumber cultivation causes nitrate accumulation in the fruit and results in food poisoning in humans; therefore, mandatory evaluation of food products becomes inevitable. Hyperspectral imaging has a very good ability to evaluate the quality of fruits and vegetables in a non-destructive manner. The goal of the present paper was to identify excess nitrogen in cucumber plants. To obtain a reliable result, the majority voting method was used, which takes into account the unanimity of five classifiers, namely, the hybrid artificial neural network-imperialism competitive algorithm (ANN-ICA), the hybrid artificial neural network-harmonic search (ANN-HS) algorithm, linear discrimination analysis (LDA), the radial basis function network (RBF), and the K-nearest-neighborhood (KNN). The wavelengths of 723, 781, and 901 nm were determined as optimal wavelengths using the hybrid artificial neural network-biogeography-based optimization (ANN-BBO) algorithm, and the performance of classifiers was investigated using the optimal spectrum. The results of a t-test showed that there was no significant difference in the precision of the algorithm when using the optimal wavelengths and wavelengths of the whole range. The correct classification rate of the classifiers ANN-ICA, ANN-HS, LDA, RBF, and KNN were 96.14%, 96.11%, 95.73%, 64.03%, and 95.24%, respectively. The correct classification rate of majority voting (MV) was 95.55% for test data in 200 iterations, which indicates the system was successful in distinguishing nitrogen-rich leaves from leaves with a standard content of nitrogen.

Identification of Internal Defects in Potato Using Spectroscopy and Computational Intelligence Based on Majority Voting Techniques.

Potatoes are one of the most demanded products due to their richness in nutrients. However, the lack of attention to external and, especially, internal defects greatly reduces its marketability and makes it prone to a variety of diseases. The present study aims to identify healthy-looking potatoes but with internal defects. A visible (Vis), near-infrared (NIR), and short-wavelength infrared (SWIR) spectrometer was used to capture spectral data from the samples. Using a hybrid of artificial neural networks (ANN) and the cultural algorithm (CA), the wavelengths of 861, 883, and 998 nm in Vis/NIR region, and 1539, 1858, and 1896 nm in the SWIR region were selected as optimal. Then, the samples were classified into either healthy or defective class using an ensemble method consisting of four classifiers, namely hybrid ANN and imperialist competitive algorithm (ANN-ICA), hybrid ANN and harmony search algorithm (ANN-HS), linear discriminant analysis (LDA), and k-nearest neighbors (KNN), combined with the majority voting (MV) rule. The performance of the classifier was assessed using only the selected wavelengths and using all the spectral data. The total correct classification rates using all the spectral data were 96.3% and 86.1% in SWIR and Vis/NIR ranges, respectively, and using the optimal wavelengths 94.1% and 83.4% in SWIR and Vis/NIR, respectively. The statistical tests revealed that there are no significant differences between these datasets. Interestingly, the best results were obtained using only LDA, achieving 97.7% accuracy for the selected wavelengths in the SWIR spectral range.

Non-Destructive Prediction of Titratable Acidity and Taste Index Properties of Gala Apple Using Combination of Different Hybrids ANN and PLSR-Model Based Spectral Data.

Non-destructive estimation of the internal properties of fruits and vegetables is very important, because better management can be provided for subsequent operations. Researchers and scientists around the world are focusing on non-destructive methods because if they are developed and commercialized, there will be an impressive change in the food industry. In this regard, this paper aims to present a non-destructive method based on Vis-NIR spectral data. The different stages of the proposed algorithm are: (1) Collection of samples of Gala apples, (2) Spectral data extraction by spectroscopy, (3) Pre-processing of spectral data, (4) Measurement of chemical properties of titratable acidity (TA) and taste index, (5) Selection of key wavelengths using hybrid artificial neural network-firefly algorithm (ANN-FA), (6) Non-destructive estimation of the properties using two methods of hybrid ANN- Particle swarm optimization algorithm and partial least squares regression. For considering the reliability of methods for estimating the chemical properties, the prediction operation was executed in 300 iterations. The results represented that the mean and standard deviation of the correlation coefficient and the root mean square error of hybrid ANN-PSO and PLSR for TA were 0.9095 ± 0.0175, 0.0598 ± 0.0064, 0.834 ± 0.0313 and 0.0761 ± 0.0061 respectively. These values for taste index were 0.918 ± 0.02, 3.2 ± 0.39, 0.836 ± 0.033 and 4.09 ± 0.403, respectively. Therefore, it can be concluded that the hybrid ANN-PSO has a better performance for non-destructive prediction of the two mentioned chemical properties than the PLSR method. In general, the proposed method can predict the chemical properties of TA and taste index non-destructively, which is very useful for mechanized harvesting and management of post-harvest operation.

Non-Destructive Estimation of Total Chlorophyll Content of Apple Fruit Based on Color Feature, Spectral Data and the Most Effective Wavelengths Using Hybrid Artificial Neural Network-Imperialist Competitive Algorithm.

Non-destructive assessment of the physicochemical properties of food products, especially fruits, makes it possible to examine the internal quality without any damage. This is applicable at different stages of fruit growth, harvesting stage, and storage as well as at the market stage. In this regard, the present study aimed to estimate the total chlorophyll content using three types of data: color data, spectral data, and spectral data related to the most effective wavelengths. The most important steps of the proposed algorithms include extracting spectral and color data from each sample of Fuji cultivar apple, selecting the most effective wavelengths at the range of 660-720 nm using hybrid artificial neural network-particle swarm optimization (ANN-PSO), non-destructive assessment of the chemical property of total chlorophyll content based on color data, and spectral data using hybrid artificial neural network-Imperialist competitive algorithm (ANN-ICA). In order to assess the reliability of the hybrid ANN-ICA, 1000 iterations were performed after selecting the optimal structure of the artificial neural network. According to the results, in the best training mode and using spectral data and the most effective wavelength, total chlorophyll content was predicted with the R2 and RMSE of 0.991 and 0.0035, 0.997 and 0.001, 0.997 and 0.0006, respectively.

An automatic visible-range video weed detection, segmentation and classification prototype in potato field.

Weeds might be defined as destructive plants that grow and compete with agricultural crops in order to achieve water and nutrients. Uniform spray of herbicides is nowadays a common cause in crops poisoning, environment pollution and high cost of herbicide consumption. Site-specific spraying is a possible solution for the problems that occur with uniform spray in fields. For this reason, a machine vision prototype is proposed in this study based on video processing and meta-heuristic classifiers for online identification and classification of Marfona potato plant (Solanum tuberosum) and 4299 samples from five weed plant varieties: Malva neglecta (mallow), Portulaca oleracea (purslane), Chenopodium album L (lamb's quarters), Secale cereale L (rye) and Xanthium strumarium (coklebur). In order to properly train the machine vision system, various videos taken from two Marfona potato fields within a surface of six hectares are used. After extraction of texture features based on the gray level co-occurrence matrix (GLCM), color features, spectral descriptors of texture, moment invariants and shape features, six effective discriminant features were selected: the standard deviation of saturation (S) component in HSV color space, difference of first and seventh moment invariants, mean value of hue component (H) in HSI color space, area to length ratio, average blue-difference chrominance (Cb) component in YCbCr color space and standard deviation of in-phase (I) component in YIQ color space. Classification results show a high accuracy of 98% correct classification rate (CCR) over the test set, being able to properly identify potato plant from previously mentioned five different weed varieties. Finally, the machine vision prototype was tested in field under real conditions and was able to properly detect, segment and classify weed from potato plant at a speed of up to 0.15 m/s.

Weed Classification for Site-Specific Weed Management Using an Automated Stereo Computer-Vision Machine-Learning System in Rice Fields.

Site-specific weed management and selective application of herbicides as eco-friendly techniques are still challenging tasks to perform, especially for densely cultivated crops, such as rice. This study is aimed at developing a stereo vision system for distinguishing between rice plants and weeds and further discriminating two types of weeds in a rice field by using artificial neural networks (ANNs) and two metaheuristic algorithms. For this purpose, stereo videos were recorded across the rice field and different channels were extracted and decomposed into the constituent frames. Next, upon pre-processing and segmentation of the frames, green plants were extracted out of the background. For accurate discrimination of the rice and weeds, a total of 302 color, shape, and texture features were identified. Two metaheuristic algorithms, namely particle swarm optimization (PSO) and the bee algorithm (BA), were used to optimize the neural network for selecting the most effective features and classifying different types of weeds, respectively. Comparing the proposed classification method with the K-nearest neighbors (KNN) classifier, it was found that the proposed ANN-BA classifier reached accuracies of 88.74% and 87.96% for right and left channels, respectively, over the test set. Taking into account either the arithmetic or the geometric means as the basis, the accuracies were increased up to 92.02% and 90.7%, respectively, over the test set. On the other hand, the KNN suffered from more cases of misclassification, as compared to the proposed ANN-BA classifier, generating an overall accuracy of 76.62% and 85.59% for the classification of the right and left channel data, respectively, and 85.84% and 84.07% for the arithmetic and geometric mean values, respectively.

A Computer Vision System Based on Majority-Voting Ensemble Neural Network for the Automatic Classification of Three Chickpea Varieties.

Since different varieties of crops have specific applications, it is therefore important to properly identify each cultivar, in order to avoid fake varieties being sold as genuine, i.e., fraud. Despite that properly trained human experts might accurately identify and classify crop varieties, computer vision systems are needed since conditions such as fatigue, reproducibility, and so on, can influence the expert's judgment and assessment. Chickpea (Cicer arietinum L.) is an important legume at the world-level and has several varieties. Three chickpea varieties with a rather similar visual appearance were studied here: Adel, Arman, and Azad chickpeas. The purpose of this paper is to present a computer vision system for the automatic classification of those chickpea varieties. First, segmentation was performed using an Hue Saturation Intensity (HSI) color space threshold. Next, color and textural (from the gray level co-occurrence matrix, GLCM) properties (features) were extracted from the chickpea sample images. Then, using the hybrid artificial neural network-cultural algorithm (ANN-CA), the sub-optimal combination of the five most effective properties (mean of the RGB color space components, mean of the HSI color space components, entropy of GLCM matrix at 90°, standard deviation of GLCM matrix at 0°, and mean third component in YCbCr color space) were selected as discriminant features. Finally, an ANN-PSO/ACO/HS majority voting (MV) ensemble methodology merging three different classifier outputs, namely the hybrid artificial neural network-particle swarm optimization (ANN-PSO), hybrid artificial neural network-ant colony optimization (ANN-ACO), and hybrid artificial neural network-harmonic search (ANN-HS), was used. Results showed that the ensemble ANN-PSO/ACO/HS-MV classifier approach reached an average classification accuracy of 99.10 ± 0.75% over the test set, after averaging 1000 random iterations.