An Open Source Simulation Toolbox for Annotation of Images and Point Clouds in Agricultural Scenarios

In recent years, the utilization of RGB cameras and LiDAR sensors in agricultural settings has surged, leading to an expanded application of machine learning techniques. Nonetheless, many machine learning challenges in agriculture are hampered by the laborious and cost-intensive process of data labeling, a task made particularly complex by the variability of crops and the sparse nature of point cloud information derived from LiDAR data. Moreover, training datasets are typically site-specific, encompassing factors such as light conditions and time of day, and often capture only a single point in a crop growing season. This specificity complicates the development of models that can generalize across different crop types, cultivars, management practices, seasons, and other variables. To address these issues, this article presents an open-source simulation toolbox designed for the easy generation of synthetic labeled data for both RGB imagery and point cloud information, applicable to a wide array of cultivars. We demonstrate how this toolbox can generate a variety of datasets with custom annotations and conditions, and we provide a straightforward pipeline for integrating this data with numerous machine learning models, specifically for this manuscript we applied an image object detection and a semantic segmentation point cloud model. This approach paves the way for a broad range of potential applications in the field of agriculture.