An RGB-D Image Dataset for Lychee Detection and Maturity Classification for Robotic Harvesting
Description
Lychee is a high-value subtropical fruit. The adoption of vision-based harvesting robots can significantly improve productivity while reduce reliance on labor. High-quality data are essential for developing such harvesting robots. However, there are currently no consistently and comprehensively annotated open-source lychee datasets featuring fruits in natural growing environments. To address this, we constructed a dataset to facilitate lychee detection and maturity classification. Color (RGB) images were acquired under diverse weather conditions, and at different times of the day, across multiple lychee varieties, such as Nuomici, Feizixiao, Heiye, and Huaizhi. The dataset encompasses three different ripeness stages and contains 11,414 images, consisting of 878 raw RGB images, 8,780 augmented RGB images, and 1,756 depth images. The images are annotated with 9,658 pairs of lables for lychee detection and maturity classification. To improve annotation consistency, three individuals independently labeled the data, and their results were then aggregated and verified by a fourth reviewer. Detailed statistical analyses were done to examine the dataset. Finally, we performed experiments using three representative deep learning models to evaluate the dataset. It is publicly available for academic use.
Files
Steps to reproduce
1.we constructed a dataset to facilitate lychee detection and maturity classification, which was collected using a self-developed multi-modal sensor module. 2.Videos were captured from one side of lychee trees at a distance of roughly 20–60 cm from the canopies, with a walking speed of approximately 1 m/s. The low-speed movement helped reduce motion-induced image blur. Data collections were conducted during the three-week peak of lychee ripening (specifically, on June 5, 10, 11, 12, and 19, 2025). Multiple common lychee varieties, such as Nuomici, Feizixiao, Heiye, and Huaizhi, were captured under various weather conditions (e.g., light rain on June 11, and sunny on other days) and at different times of day (morning, noon, and evening). The ROS plugin (bag_to_images) was used to convert the recorded .bag files into image sequences. One image was extracted for every 10 frames in the video. To improve dataset diversity and reduce redundancy, after the first image was manually selected in a sequence, the Structural Similarity Inde was employed to select the most dissimilar frames within each 10-frame segment from the previous selected image.