Key Point-Based Orientation Estimation of Strawberries for Robotic Fruit Picking

Selective robotic harvesting can help address labour shortages affecting modern global agriculture. For an accurate and efficient picking process, a robotic harvester requires the precise location and orientation of the fruit to effectively plan the trajectory of the end effector. The current methods for estimating fruit orientation employ either complete 3D information registered from multiple views or rely on fully-supervised learning techniques, requiring difficult-toobtain manual annotation of the reference orientation. In this paper, we introduce a novel key-point-based fruit orientation estimation method for the prediction of 3D orientation from 2D images directly. The proposed technique can work without full 3D orientation annotations but can also exploit such information for improved accuracy. We evaluate our work on two separate datasets of strawberry images obtained from real-world scenarios. Our method achieves state-of-the-art performance with an average error as low as 8 degrees, improving predictions by similar to 30% compared to previous work presented in [17]. Furthermore, our method is suited for real-time robotic applications with fast inference times of similar to 30 ms.