The number of spikes per unit area is a key yield component for cereal crops such as wheat, which is popularly used in wheat research for crop improvement. With the fast maturity of smartphone imaging hardware and recent advances in image processing and lightweight deep learning techniques, it is possible to acquire high-resolution images using a smartphone camera, followed by the analysis of wheat spikes per unit area through pre-trained artificial intelligence algorithms. Then, by combining detected spike number with variety-based spikelet number and grain weight, it is feasible to carry out a near real-time estimation of yield potential for a given wheat variety in the field. This AI-driven approach becomes more powerful when a range of varieties are included in the training datasets, enabling an effective and valuable approach for yield-related studies in breeding, cultivation, and agricultural production. In this study, we present a novel smartphone-based software application that combines smartphone imaging, lightweight and embedded deep learning, with yield prediction algorithms and applied the software to wheat cultivation experiments. This open-source Android application is called YieldQuant-Mobile (YQ-M), which was developed to measure a key yield trait (i.e. spikes per unit area) and then estimate yield based on the trait. Through YQ-M and smartphones, we standardized the in-field imaging of wheat plots, streamlined the detection of spikes per unit area and the prediction of yield, without a prerequisite of in-field WiFi or mobile network. In this article, we introduce the YQ-M in detail, including: 1) the data acquisition designed to standardize the collection of wheat images from an overhead perspective using Android smartphones; 2) the data pre-processing of the acquired image to reduce the computational time for image analysis; 3) the extraction of wheat spike features through deep learning (i.e. YOLOV4) and transfer learning; 4) the application of TensorFlow.lite to transform the trained model into a lightweight MobileNetV2-YOLOV4 model, so that wheat spike detection can be operated on an Android smartphone; 5) finally, the establishment of a mobile phone database to incorporate historic datasets of key yield components collected from different wheat varieties into YQ-M using Android SDK and SQLite. Additionally, to ensure that our work could reach the broader research community, we developed a Graphical User Interface (GUI) for YQ-M, which contains: 1) the spike detection module that identifies the number of wheat spikes from a smartphone image; 2) the yield prediction module that invokes near real-time yield prediction using detected spike numbers and related parameters such as wheat varieties, place of production, accumulated temperature, and unit area. During our research, we have tested YQ-M with 80 representative varieties (240 one-square-meter plots, three replicates) selected from the main wheat producing areas in China. The computed accuracy, recall, average accuracy, and F1-score for the learning model are 84.43%, 91.05%, 91.96%, and 0.88, respectively. The coefficient of determination between YQ-M predicted yield values and post-harvest manual yield measurement is 0.839 (n=80 varieties, P<0.05; Root Mean Square Error=17.641 g/m2). The results suggest that YQ-M presented here has a high accuracy in the detection of wheat spikes per unit area and can produce a consistent yield prediction for the selected wheat varieties under complex field conditions. Furthermore, YQ-M can be easily accessed and expanded to incorporate new varieties and crop species, indicating the usability and extendibility of the software application. Hence, we believe that YQ-M is likely to provide a step change in our abilities to analyze yield-related components for different wheat varieties, a low-cost, accessible, and reliable approach that can contribute to smart breeding, cultivation and, potentially, agricultural production. © 2021, Editorial Department of the Transactions of the Chinese Society of Agricultural Engineering. All right reserved.
