Affiliations: [a] Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), Jiangnan University, Wuxi, Jiangsu, China | [b] Department of Bioresource Engineering, Macdonald Campus, McGill University, Sainte-Anne-de-Bellevue, QC, Canada
Correspondence:
[*]
Corresponding authors: Shangpeng Sun, Department of Bioresource Engineering, McGill University, Macdonald Stewart Building, 21111 Lakeshore Road, Sainte-Anne-de-Bellevue, QC, Canada. E-mail: shangpeng.sun@mcgill.ca. Qibing Zhu, Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), Jiangnan University, 1800 Lihu Avenue, Wuxi, Jiangsu 214122, China. E-mail: zhuqib@163.com.
Abstract: The first major contribution of the paper is the proposal of using an improved DEtection Transformer network (named R2N-DETR) and Kinect-V2 camera for detecting multiple-size peaches under orchards with varied illumination and fruit occlusion. R2N-DETR model first employed Res2Net-50 to extract a fused low-high level feature map containing fine spatial features and precise semantic information of multi-size peaches from Red-Green-Blue-Depth (RGB-D) images. Second, the encoder-decoder was performed on the feature map to obtain the global context. Finally, all detected objects were detected according to each object’s global context. For the detection of 1101 RGB-D images (imaged from two orchards over three years), the R2N-DETR model achieves an average precision of 0.944 and an average detecting time of 53 ms for each image. The developed system could provide precise visual guidance for robotic picking and contribute to improving yield prediction by providing accurate fruit counting.
Keywords: Deep learning, peach detection, RGB-D image, R2N-DETR, open orchard
