Research on the vision system of lychee picking robot based on stereo vision

In this study, we propose a development scheme of litchi robot vision system based on stereo vision. The system includes three components: image acquisition, image analysis, and image processing. The obtained image can be corrected according to the light environment conditions with the corresponding algorithm for image color, calculate the Hue color layer, analyze the external characteristics and positional data (2D) of litchi, obtain the 3D coordinate information of fruits and vegetables, and provide to the running device for route planning and vegetable and fruit picking claw tools for picking. The integration test results for the robot arm and claw tool demonstrate the following: (1) Using image acquisition, the system can acquire high-quality images in real-time via a combination of automatic exposure and white balance conversion. (2) Using image analysis techniques such as background separation, edge detection, and external feature search, it is possible to successfully calculate the location coordinates of the litchi's tip, center, and bottom. (3) Using image processing, the claw of the picking machine can determine the condition of overlapping lychees and their state of ripening based on the depth information of feature points used in the picking operation. (4) Under the working range of 18.4 cm x 33.6 cm, the success rate of picking litchi with a diameter of 28 mm, 34 mm, and 42 mm was 93.3%, 91.1 %, and 84.4%, respectively, exceeding the design expectation level.