Original papers
Multispectral image based germination detection of potato by using supervised multiple threshold segmentation model and Canny edge detector

https://doi.org/10.1016/j.compag.2021.106041Get rights and content

Highlights

  • The multispectral imaging system was used for potato germination detection.

  • The SMTSM combined with Canny edge detector was proposed to detect germinated area.

  • The SMTSM was constructed by genetic programming and hybrid fitness function.

  • The GP images and their optimal segmentation masks were obtained by the SMTSM.

  • Effectiveness of the proposed method was verified by edible and breeding potatoes.

Abstract

Whether from the perspective of agricultural production or food safety, potato germination detection is of great significance. Since the features (color, texture and context) of the germination area are similar to those of the non-germination area, the existing vision frameworks are difficult to accurately detect the germinations on the surface of potatoes. In this study, the method for detecting potato germination based on multispectral image combined with supervised multiple threshold segmentation model (SMTSM) and Canny edge detector was proposed. The SMTSM based on Genetic Programming algorithm combined with a hybrid fitness function (HF-GP) was utilized to transform the original multispectral images into multiple 2-D images for improving the contrast between region of interest (ROI) and background. A sub-mask of each transformed image was constructed using optimal segmentation threshold, and all of sub-masks were merged through pixel-multiplication to obtain segmentation mask. Meanwhile, in order to filter out the boundless areas that are misidentified as germinations, Canny edge detector was used on gray image to obtain edge mask. Finally, the segmentation mask and the edge mask were combined to complete the detection of germination of potato. Experimental results shown that the proposed method achieved the TPR of 90.91% and the precision of 89.28% for the edible potatoes, which were 4.17–19.05% and 12.39–24.62% higher than the competitive detectors in TPR and precision respectively. For the breeding potatoes, the proposed method with 89.67% of TPR and 86.37% of precision was 9.74–24.58% and 15.70–20.39% better than the competitors in TPR and precision respectively. The comparison confirms the proposed method has excellent detection effect on potato’s germination.

Introduction

The potato, one of the “six health foods” recognized by the United National Food Agricultural Organization (FAO), is cultivated in more than 150 counties around the world. Potatoes are prone to germination during storage, and germinated potatoes contain toxins such as solanine, which brings potential food safety risks (Lu et al., 2019). For potato growers, they need to carefully select potato with germination as seed for ensuring yield (Cho et al., 2017). Therefore, whether from the perspective of agricultural production or food safety, development of the detection methods to obtain potato germination is demand.

With the expensive and inefficient of the labor resource, the detection of the external quality for agriculture products (including fruit and vegetable) requires transformation from traditional manual visual inspection to technology-based detection (Hameed, et al., 2018). Machine vision technology is built on imaging technique and image processing algorithm, which has been widely used in the external quality evaluation of agricultural products, such as defect detection (Rong et al., 2017) and quality sorting (Ramirez-Paredes and Hernandez-Belmonte, 2020). Hence, the purpose of this research is to develop an effective and efficient machine vision framework to detect the potato germinations from two aspects: imaging system selection and image processing method design.

Currently, there are mainly two types of imaging systems (i.e. RGB imaging system and hyperspectral/multispectral imaging system) used to acquire the image of agricultural products’ surface. RGB images can provide abundant color, texture and context features, allowing machine vision to evaluate the quality of fruits and vegetables (Vibhute and Bodhe, 2012, Yao et al., 2017). However, due to the limited spectral information, the RGB imaging system is usually applied to the detection of defects with obvious color, geometric or texture differences from normal tissues. Compared with RGB imaging system, hyperspectral imaging (HSI) system is a more popular tool for detecting and evaluating the quality and safety of agricultural products (such as apple, citrus, cucumber and tomato) due to integrated visualization and spectral information of objects (Wu et al., 2016, Lu et al., 2020). Meanwhile, the machine vision framework with HSI system was widely used to examine the external qualities of potatoes (López-Maestresalas et al., 2016, Sanchez et al., 2020). HSI system usually collects dozens or hundreds of images with continuous wavelengths, resulting in huge time costs in data storage and processing; therefore, this system is difficult to apply to the situations that require rapid detection. In order to meet the needs of rapid detection, a variety of multispectral imaging (MSI) systems have been developed. Among them, the novel MSI based on single shot method, that can simultaneously record the spatial and spectral information with one exposure, is considered to be a potential tool for online inspection of agricultural products (Gao et al., 2018, Li et al., 2019c). MSI based on single shot method has been used in fast shooting of agricultural products, such as real-time defects detection (Abdelsalam and Sayed, 2016), online quality assessment (Khodabakhshian et al., 2016) and disease detection (Kerkech et al., 2020). Zhang et al. (2019) applied MSI for detection and classification of potato defects, and achieved 90.70% classification accuracy for normal and six types of defect. Considering the need of detection speed and sufficient information acquisition, MSI based on single shot method was used to capture potato’s image in this research.

Using appropriate image processing algorithms to detect (segment) the region of interest (ROI) from background is another key issue in defect detection of agricultural products based on machine vision technology. Many image processing algorithms, such as threshold segmentation (Zhang et al., 2019), watershed segmentation (Luo et al., 2019), segmentation based on pixel classifier (Ebrahimi et al., 2017), were proposed to detect ROI of object. Because threshold segmentation has the advantages of simple principle and high time-efficiency, it was wildly used in ROI detection of agricultural products (Riehle et al., 2020). To obtain the ROI of apple’s surface, Mizushima and Lu (2013) transformed the raw RGB image into grayscale image by using linear support vector machine (SVM), and conducted Otsu’s method on this grayscale image. The above research reported an image processing method combining image transformation and threshold segmentation method. This type of method first convert the original image to a new feature space, so that the ROI and the background have a higher contrast in the feature space, thereby facilitating the determination of segmentation threshold value and improving the effect of segmentation (Zhang et al., 2018, Li et al., 2019a). Recently, various HSI image-based machine vision frameworks employed this type of method to extract ROI. Alam et al. (2018) firstly selected three wavelengths from HSI image and then employed principal component analysis (PCA) to transform the three wavelengths into the best principal components (PCs). Finally, the authors conducted a global thresholding method on one PC to detect defects on the surface of apples. Zhang et al. (2020) employed band ratio method and PCA algorithm to convert the HSI data into two different gray images, then, a simple threshold method was conducted on the image fused by these two gray images for defects detection of mandarins. Zhang et al. (2019) selected images at characteristic wavelengths from HSI data, through empirical observation, to construct a band ratio image, and completed ROI segmentation on band ratio image by using manually determined threshold. The above researches show that PCA and band ratio methods are usually used as transformation methods. However, PCA is a linear transformation model, its ability to improve image contrast is limited by the model performance. Although the band ratio method is a non-linear transformation method, selecting characteristic wavebands is still a tough task. More importantly, all of above-mentioned methods performed image transformation and threshold segmentation as two separate algorithms, so it is difficult to ensure that the image transformation method matches the subsequent threshold segmentation method well. The above reasons make that existing segmentation methods are difficult to achieve satisfactory detection result for potato germination, the colors or texture features of which are similar to the normal region or other defects (such as, common scab, bruise) on potato’s surface. Therefore, it is urgent to find an algorithm suitable for potato germination detection.

Motivated by the above observations, this research developed a detection method combined two sub-detectors for potato germination based on the MSI. The major contributions are summarized as follows.

  • (1)

    The MSI based on single shot method is employed to capture both spectral and spatial information of potatoes.

  • (2)

    The genetic programming algorithm, with a hybrid fitness function containing feature correlation and pixel classification accuracy (HF-GP), is used to generate supervised multiple threshold segmentation models (SMTSM), each of which includes one waveband operation and one corresponding threshold pair. Multiple waveband operations are used to non-linearly transform multispectral (MS) images into multiple feature images (called GP image), and the optimal threshold pairs are separately conducted on the corresponding GP images to extract the ROIs.

  • (3)

    The Canny edge detector is employed based on the spatial feature to decrease the false positive problem resulting from the uneven light intensity distribution of the potato.

Experiments based on edible and breeding potato samples shown that performance of the proposal method is significantly better than that of the existing methods based on MS image.

Section snippets

Potato samples

To fully evaluate the performance of the proposed method, two types of potato, namely, edible potato and breeding potato were used in this study. The edible potato and breeding potato purchased from Chaoyang Farmers’ Market and Vegetable Seed Company in Wuxi, China, respectively. The size of the edible potato samples is approximately 55–75 mm in diameter and 80–120 mm in length, while the breeding potato samples are about 48–52 mm in diameter and 48–64 mm in length. From Fig. 1, the regions of

Results and discussion

This section has four sections. Section 3.1 compared the multiple with single threshold segmentation model. Section 3.2 discussed the results of five segmentation methods based on spectrum of MS image. Several examples were cited in Section 3.3 to illustrate the role of the sub-detector based on Canny edge detector. Finally, Section 3.4 presented the germination detection results using five fused methods. The comparative results demonstrated that our proposed detector outperforms the other four

Conclusions

In this research, a novel and effective framework of machine vision technology for the detection of potato germination is presented. This technology employs MSI based on single shot method to quickly capture the 25-waveband spectral and spatial information of the potatoes. Furthermore, the detection method combined with SMTSM (sub-detector 1) and Canny edge detector (sub-detector 2) is proposed to detect germinations on potatoes based on MS image. The SMTSM obtained by HF-GP converts the raw MS

CRediT authorship contribution statement

Yu Yang: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing - original draft. Xin Zhao: Investigation, Resources, Data curation, Writing - original draft. Min Huang: Writing - review & editing, Formal analysis, Supervision, Visualization. Xin Wang: Investigation, Resources, Data curation, Writing - original draft. Qibing Zhu: Writing - review & editing, Formal analysis, Supervision, Visualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

Dr. Qibing Zhu and Dr. Min Huang gratefully acknowledge the financial support from the National Natural Science Foundation of China (Grant no. 61772240, 61775086), the 111 Project (B12018).

References (33)

Cited by (0)

View full text