VisText-Mosquito: A Multimodal Dataset for Mosquito Breeding Site Detection, Surface Segmentation, and Reasoning
Description
VisText-Mosquito is a comprehensive multimodal dataset designed to support the detection of mosquito breeding sites, segmentation of water surfaces, and generation of natural language reasoning for explainable AI applications. It consists of three core components: Breeding Place Detection: This part includes 1,828 images with 3,752 annotations across five classes: Coconut-Exocarp, Vase, Tire, Drain-Inlet, and Bottle. The images were collected from diverse urban, semi-urban, and rural environments in Bangladesh under daylight conditions to ensure visual consistency. Detection performance was validated using state-of-the-art object detection models, including YOLOv5s, YOLOv8n, and YOLOv9s, with YOLOv9s achieving the highest mAP@50. Water Surface Segmentation: This component contains 142 images with 253 annotations across two classes: \texttt{vase_with_water} and \texttt{tire_with_water}. YOLOv8x-Seg and YOLOv11n-Seg models were used to validate segmentation performance in detecting water surfaces within the identified containers. Textual Reasoning Generation: Each image is linked with a natural language reasoning statement that explains the presence or absence of breeding risk. A fine-tuned BLIP model was used to generate these explanations, achieving strong performance on BLEU, BERTScore, and ROUGE-L metrics. The VisText-Mosquito dataset offers a novel multimodal benchmark for training and evaluating AI models that combine detection, segmentation, and interpretability. It serves as a valuable resource for researchers and public health professionals aiming to develop explainable, scalable mosquito control solutions.
Files
Steps to reproduce
The directory paths for training, validation, and testing datasets are specified in a configuration file named "data.yaml", ensuring that anyone replicating the study can easily locate and use the data.