SDT_mars_custom_dataset
Description
The dataset is composed of high fidelity images of a physical planetary analog environment that are annotated with foreground and background semantic labels and were created for training lightweight semantic segmentation models (e.g., SegFormer) for autonomous rover navigation and ingestion of the Semantic Digital Twin (SDT). To help close the gap between the simulation and reality, a physical analog "sandbox" test bed was built to simulate both Martian terrain and hazard terrain, with dynamic obstacles, different soil properties and rock structures. An edge-computing setup (Raspberry Pi 4B with a Sony IMX708 Camera Module 3) captured a continuous data stream, resulting in 500 images being taken as raw data. The out-of-focus and redundant frames were rejected and the final baseline consisted of 482 high data quality images. These images have been uploaded to the Roboflow platform where pixel-perfect, polygon-based semantic segmentation masks have been handcrafted for each image, for a specific terrain class and hazard class. Semantic Classes (6-Class Taxonomy): The dataset is fully annotated across the following 6 distinct terrain and hazard categories: Rock Sand Ridge Hole Bedrock Soil The most efficient way to split a dataset into three or more segments. The dataset was thoroughly split into 70% training, 15% validation and 15% test sets to prevent data leakage and provide a strong evaluation. Training Set (70%): ~337 images Validation Set (15%): ~72 images Testing Set (15%): ~72 images Data Augmentation: The data augmentation was applied only to the 70% Training Set, in order to improve the generalization performance of the neural network, while keeping the evaluation metrics as they are. These training frames were then augmented by a 3x factor by methods like random rotation, cropping and contrast changes. This augmented training data was then re-combined with the strictly un-augmented validation and testing sets, resulting in a total of ~1,155 images ready to be used in edge-AI training pipelines. Mixed_dataset_70_30: In addition to the primary custom dataset described above, this repository also includes a Mixed_dataset_70_30 archive. This represents a 70-30 domain-blended mixture, created by combining our ~1,155 custom augmented images with randomly sampled frames from the public S5Mars source domain dataset. This blended dataset is provided alongside the custom dataset to ensure 100% reproducibility of the domain-mixture experiments and results discussed in our associated research. Potential Applications: Lightweight training of Vision Transformers and CNNs for semantic segmentation. Planetary micro rover Hardware-in-the-loop testing. Testing of bandwidth optimization and telemetry throttling, and research for deep-space communications.