Real-time recognition of on-branch olive ripening stages by a deep convolutional neural network

Olive in its various ripening stages is an important agricultural product, especially for its oil. Early detection of ripening stages of on-branch olives provides an insight into suitable site-specific managements. To fulfill these requirements, a model has been developed in the current study. This model is based on RGB images and deep Convolutional Neural Networks (CNN). Two Iranian olive cultivars (Zard and Roghani) in four ripening stages (immature, green maturity, black maturity, and fully mature) were considered (total eight classes). No structured imaging system was considered to ensure a natural condition of the experiment and generality of the method. The proposed model was selected based upon testing several popular or developed CNNs, and according to the criterion such as accuracy, prediction time, and computational burden. Multiple layers of Convolution, Max-Pooling, and Batch Normalization, followed by a Global Average Pooling layer (GAP), were contained in the proposed model. For this network, six different optimizers, i.e., Adagrad, SGD, SGDM, RMSProp, Adam, and Nadam, were evaluated, among which Nadam showed the best efficiency. According to the results, the overall accuracy of the proposed model was 91.91 %, and it performed the processing of a single frame in only 12.64 ms on CPU. It reflects the real-time potential and the robustness of the model to classify on-branch olives based on their ripening stages. If an orchard consists of just Zard cultivar or both the Zard and Roghani cultivars, the unripe Zard cultivar, among all eight classes, will be recognized with 100 % accuracy. However, if the Roghani cultivar is only considered, the over-ripening stage will be recognized with the highest accuracy. The results of this study showed that the proposed model could be effectively embedded in a system to treat olive trees based on the conditions of a particular branch in precision horticulture.