Dataset
Description
This research investigates the hypothesis that integrating complementary sensing modalities—computer vision, radar-based sensing, and emotion-aware behavioral analysis—within a unified decision-level fusion framework can significantly improve crime detection performance compared to individual sensing approaches. The study further hypothesizes that an interpretable fuzzy logic fusion mechanism can effectively combine heterogeneous sensor outputs while reducing uncertainty and false alarms in real-world surveillance scenarios. The dataset contains the intermediate outputs and final results generated by the three sensing arms of the proposed multimodal crime detection framework. The vision component was developed using a YOLO-based object detection model to identify visible threats, suspicious objects, and contextual scene information from surveillance imagery. The radar component analyzes metallic signatures and motion-related features to estimate the likelihood of concealed threats using radar cross-section characteristics and motion behavior indicators. The emotion recognition component evaluates facial expressions and behavioral cues to infer potential intent and emotional states associated with suspicious activities. Data were collected through a combination of publicly available datasets, controlled experimental scenarios, and simulated radar measurements designed to represent realistic surveillance conditions. Each sensing modality produces a normalized threat score ranging from 0 to 1. These scores are subsequently processed through a fuzzy logic-based decision-level fusion engine that generates a final threat assessment categorized into threat levels such as SAFE, SUSPICIOUS, and DANGEROUS. The dataset includes raw modality outputs, normalized threat scores, fusion results, evaluation metrics, graphical outputs, and supporting configuration files. The results demonstrate that multimodal fusion improves overall detection reliability, increasing ROC AUC from approximately 0.89 to 0.96 while reducing false alarm rates by approximately 17%. The data further illustrate how complementary sensing information can resolve ambiguity that may exist within individual modalities. Researchers may use this dataset to investigate multimodal sensor fusion techniques, fuzzy inference systems, threat assessment algorithms, surveillance analytics, explainable artificial intelligence, and decision-support systems. The provided files enable replication of the fusion process, evaluation of individual modality contributions, and comparison with alternative fusion strategies.
Files
Steps to reproduce
This dataset was developed to evaluate a multimodal crime detection framework that combines computer vision, radar sensing, and emotion recognition within a fuzzy logic-based decision-level fusion system. The vision arm uses a YOLO-based detector to identify visible threats and generate confidence-based threat scores. The radar arm analyzes metallic signatures and motion characteristics to estimate concealed threats, while the emotion arm employs deep learning to classify facial emotions and infer behavioral intent. Each modality produces a normalized threat score that is fused using a Mamdani fuzzy inference system implemented in Python and MATLAB/Simulink. The dataset includes modality outputs, fusion results, evaluation metrics, and visualization files. Experimental results demonstrate that multimodal fusion improves detection reliability, achieving an ROC AUC increase from approximately 0.89 to 0.96 while reducing false alarms by about 17%. The workflow can be reproduced using the provided scripts, configuration files, and evaluation outputs, enabling further research in sensor fusion, intelligent surveillance, explainable AI, and crime detection systems.