Extraction of soybean coverage from UAV images combined with 3D dense point cloud

An accurate extraction of soybean coverage cannot be gained only by using two-dimensional (2D) remote sensing images with complex background interference, such as weeds. In this study, a novel extraction was proposed for the soybean coverage under a complex background, combining the visible light spectrum and three-dimensional (3D) dense point clouds. The improved Structure from Motion (SfM) and Semi-Global Matching (SGM) were used to extract the high-precision and dense RGB 3D point clouds from the Unmanned Aerial Vehicle (UAV) overlapping images. A gamma-enhanced visible light vegetation index was explored to extract the vegetation information. A local threshold segmentation was selected with the best structural element to eliminate the low weeds and other noises of non-soybean crops. A field experiment was carried out using the visible light UAV data of soybean planting areas under the different periods, the weed mixing, and topographic fluctuation. The 3D dense point cloud was then selected to generate the irregular triangular network and digital surface grid in the two study areas, where the height difference between crops and background objects was mapped into the gray difference of the grid map. The local threshold function was obtained through the top-hat transform and Otsu method, where the digital surface grid was binarized to accurately separate the crop from the background under the uneven gray distribution. Furthermore, the 3D dense point cloud and intersection analysis were combined to filter the low weeds and non-soybean vegetation. The crop and background samples were manually marked in the orthophoto that derived from UAV photogrammetry. As such, the confusion matrix was constructed to evaluate the classification accuracy of the extraction. It was found that the overall classification accuracies (OA) of soybean coverage extraction were 99.30% and 98.75% in study areas 1 and 2, respectively, where the Kappa coefficients were 0.99 and 0.98, respectively. A comparative analysis was also made to verify the effectiveness and applicability of coverage extraction using 2D image segmentation, including Support Vector Machine (SVM), lab-Kmeans segmentation, bimodal threshold method, excess green index (ExGI), Color Index of Vegetation Extraction (CIVE), and Green Red Ratio Index (GRVI). The experimental results showed that the Gamma Enhanced Vegetation Index (EVI) significantly improved the accuracy of vegetation extraction, where the extraction accuracy of soybean coverage reached more than 98% using the coupled 3D point cloud, which at least 68% higher than that of the support vector machine, Lab-Kmeans segmentation, and bimodal threshold method. It infers that the Gamma EVI can be widely expected to enhance the characteristics of vegetation, while suppressing the irrelevant background, indicating excellent feasibility. In addition, the digital surface grid with 3D information can be used to effectively eliminate the complex background, indicating the strong robustness to the light change and terrain fluctuation. A large-area soybean coverage can be rapidly and accurately achieved to relieve the complex background interference, such as image resolution, uneven illumination, and topographic relief, indicating a better extraction performance using the visible light spectrum and 3D dense point clouds, compared with the 2D. The findings can provide promising technical support to the fine management and yield estimation in farmland. © 2022, Editorial Department of the Transactions of the Chinese Society of Agricultural Engineering. All right reserved.