Original papersMultispectral image based germination detection of potato by using supervised multiple threshold segmentation model and Canny edge detector
Introduction
The potato, one of the “six health foods” recognized by the United National Food Agricultural Organization (FAO), is cultivated in more than 150 counties around the world. Potatoes are prone to germination during storage, and germinated potatoes contain toxins such as solanine, which brings potential food safety risks (Lu et al., 2019). For potato growers, they need to carefully select potato with germination as seed for ensuring yield (Cho et al., 2017). Therefore, whether from the perspective of agricultural production or food safety, development of the detection methods to obtain potato germination is demand.
With the expensive and inefficient of the labor resource, the detection of the external quality for agriculture products (including fruit and vegetable) requires transformation from traditional manual visual inspection to technology-based detection (Hameed, et al., 2018). Machine vision technology is built on imaging technique and image processing algorithm, which has been widely used in the external quality evaluation of agricultural products, such as defect detection (Rong et al., 2017) and quality sorting (Ramirez-Paredes and Hernandez-Belmonte, 2020). Hence, the purpose of this research is to develop an effective and efficient machine vision framework to detect the potato germinations from two aspects: imaging system selection and image processing method design.
Currently, there are mainly two types of imaging systems (i.e. RGB imaging system and hyperspectral/multispectral imaging system) used to acquire the image of agricultural products’ surface. RGB images can provide abundant color, texture and context features, allowing machine vision to evaluate the quality of fruits and vegetables (Vibhute and Bodhe, 2012, Yao et al., 2017). However, due to the limited spectral information, the RGB imaging system is usually applied to the detection of defects with obvious color, geometric or texture differences from normal tissues. Compared with RGB imaging system, hyperspectral imaging (HSI) system is a more popular tool for detecting and evaluating the quality and safety of agricultural products (such as apple, citrus, cucumber and tomato) due to integrated visualization and spectral information of objects (Wu et al., 2016, Lu et al., 2020). Meanwhile, the machine vision framework with HSI system was widely used to examine the external qualities of potatoes (López-Maestresalas et al., 2016, Sanchez et al., 2020). HSI system usually collects dozens or hundreds of images with continuous wavelengths, resulting in huge time costs in data storage and processing; therefore, this system is difficult to apply to the situations that require rapid detection. In order to meet the needs of rapid detection, a variety of multispectral imaging (MSI) systems have been developed. Among them, the novel MSI based on single shot method, that can simultaneously record the spatial and spectral information with one exposure, is considered to be a potential tool for online inspection of agricultural products (Gao et al., 2018, Li et al., 2019c). MSI based on single shot method has been used in fast shooting of agricultural products, such as real-time defects detection (Abdelsalam and Sayed, 2016), online quality assessment (Khodabakhshian et al., 2016) and disease detection (Kerkech et al., 2020). Zhang et al. (2019) applied MSI for detection and classification of potato defects, and achieved 90.70% classification accuracy for normal and six types of defect. Considering the need of detection speed and sufficient information acquisition, MSI based on single shot method was used to capture potato’s image in this research.
Using appropriate image processing algorithms to detect (segment) the region of interest (ROI) from background is another key issue in defect detection of agricultural products based on machine vision technology. Many image processing algorithms, such as threshold segmentation (Zhang et al., 2019), watershed segmentation (Luo et al., 2019), segmentation based on pixel classifier (Ebrahimi et al., 2017), were proposed to detect ROI of object. Because threshold segmentation has the advantages of simple principle and high time-efficiency, it was wildly used in ROI detection of agricultural products (Riehle et al., 2020). To obtain the ROI of apple’s surface, Mizushima and Lu (2013) transformed the raw RGB image into grayscale image by using linear support vector machine (SVM), and conducted Otsu’s method on this grayscale image. The above research reported an image processing method combining image transformation and threshold segmentation method. This type of method first convert the original image to a new feature space, so that the ROI and the background have a higher contrast in the feature space, thereby facilitating the determination of segmentation threshold value and improving the effect of segmentation (Zhang et al., 2018, Li et al., 2019a). Recently, various HSI image-based machine vision frameworks employed this type of method to extract ROI. Alam et al. (2018) firstly selected three wavelengths from HSI image and then employed principal component analysis (PCA) to transform the three wavelengths into the best principal components (PCs). Finally, the authors conducted a global thresholding method on one PC to detect defects on the surface of apples. Zhang et al. (2020) employed band ratio method and PCA algorithm to convert the HSI data into two different gray images, then, a simple threshold method was conducted on the image fused by these two gray images for defects detection of mandarins. Zhang et al. (2019) selected images at characteristic wavelengths from HSI data, through empirical observation, to construct a band ratio image, and completed ROI segmentation on band ratio image by using manually determined threshold. The above researches show that PCA and band ratio methods are usually used as transformation methods. However, PCA is a linear transformation model, its ability to improve image contrast is limited by the model performance. Although the band ratio method is a non-linear transformation method, selecting characteristic wavebands is still a tough task. More importantly, all of above-mentioned methods performed image transformation and threshold segmentation as two separate algorithms, so it is difficult to ensure that the image transformation method matches the subsequent threshold segmentation method well. The above reasons make that existing segmentation methods are difficult to achieve satisfactory detection result for potato germination, the colors or texture features of which are similar to the normal region or other defects (such as, common scab, bruise) on potato’s surface. Therefore, it is urgent to find an algorithm suitable for potato germination detection.
Motivated by the above observations, this research developed a detection method combined two sub-detectors for potato germination based on the MSI. The major contributions are summarized as follows.
- (1)
The MSI based on single shot method is employed to capture both spectral and spatial information of potatoes.
- (2)
The genetic programming algorithm, with a hybrid fitness function containing feature correlation and pixel classification accuracy (HF-GP), is used to generate supervised multiple threshold segmentation models (SMTSM), each of which includes one waveband operation and one corresponding threshold pair. Multiple waveband operations are used to non-linearly transform multispectral (MS) images into multiple feature images (called GP image), and the optimal threshold pairs are separately conducted on the corresponding GP images to extract the ROIs.
- (3)
The Canny edge detector is employed based on the spatial feature to decrease the false positive problem resulting from the uneven light intensity distribution of the potato.
Experiments based on edible and breeding potato samples shown that performance of the proposal method is significantly better than that of the existing methods based on MS image.
Section snippets
Potato samples
To fully evaluate the performance of the proposed method, two types of potato, namely, edible potato and breeding potato were used in this study. The edible potato and breeding potato purchased from Chaoyang Farmers’ Market and Vegetable Seed Company in Wuxi, China, respectively. The size of the edible potato samples is approximately 55–75 mm in diameter and 80–120 mm in length, while the breeding potato samples are about 48–52 mm in diameter and 48–64 mm in length. From Fig. 1, the regions of
Results and discussion
This section has four sections. Section 3.1 compared the multiple with single threshold segmentation model. Section 3.2 discussed the results of five segmentation methods based on spectrum of MS image. Several examples were cited in Section 3.3 to illustrate the role of the sub-detector based on Canny edge detector. Finally, Section 3.4 presented the germination detection results using five fused methods. The comparative results demonstrated that our proposed detector outperforms the other four
Conclusions
In this research, a novel and effective framework of machine vision technology for the detection of potato germination is presented. This technology employs MSI based on single shot method to quickly capture the 25-waveband spectral and spatial information of the potatoes. Furthermore, the detection method combined with SMTSM (sub-detector 1) and Canny edge detector (sub-detector 2) is proposed to detect germinations on potatoes based on MS image. The SMTSM obtained by HF-GP converts the raw MS
CRediT authorship contribution statement
Yu Yang: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing - original draft. Xin Zhao: Investigation, Resources, Data curation, Writing - original draft. Min Huang: Writing - review & editing, Formal analysis, Supervision, Visualization. Xin Wang: Investigation, Resources, Data curation, Writing - original draft. Qibing Zhu: Writing - review & editing, Formal analysis, Supervision, Visualization.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
Dr. Qibing Zhu and Dr. Min Huang gratefully acknowledge the financial support from the National Natural Science Foundation of China (Grant no. 61772240, 61775086), the 111 Project (B12018).
References (33)
- et al.
Vision-based pest detection based on SVM classification method
Comput. Electron. Agric.
(2017) - et al.
A comprehensive review of fruit and vegetable classification techniques
Image Vis. Comput.
(2018) - et al.
Vine disease detection in UAV multispectral images using optimized image registration and deep learning segmentation approach
Comput. Electron. Agric.
(2020) - et al.
Early detection of decay on apples using hyperspectral reflectance imaging combining both principal component analysis and improved watershed segmentation method
Postharvest Biol. Technol.
(2019) - et al.
Detection of early decayed oranges based on multispectral principal component image combining both bi-dimensional empirical mode decomposition and watershed segmentation method
Postharvest Biol. Technol.
(2019) - et al.
Non-destructive detection of blackspot in potatoes by Vis-NIR and SWIR hyperspectral imaging
Food Control
(2016) - et al.
Hyperspectral imaging technology for quality and safety evaluation of horticultural products: A review and celebration of the past 20-year progress
Postharvest Biol. Technol.
(2020) - et al.
An image segmentation method for apple sorting and grading using support vector machine and Otsu’s method
Comput. Electron. Agric.
(2013) - et al.
Visual quality assessment of malting barley using color, shape and texture descriptors
Comput. Electron. Agric.
(2020) - et al.
Robust index-based semantic plant/background segmentation for RGB- images
Comput. Electron. Agric.
(2020)