Abductees-Rescue: 3D Hand Landmarks Dataset for Real-Time Abduction Detection in Surveillance Systems
Description
This work has been comprised of creating a 3D hand landmarks dataset for real-time detection of abduction cases through surveillance systems using the MediaPipe Hands framework. The dataset has been represented in two categories of 3D hand landmark gestures: normal hand gestures labeled with a "0" classlable and specific gestures indicating potential abduction cases labeled with a "1" classlable. The 3D hand landmarks have been extracted after applying MediaPipe on 4545 videos of normal hand gestures and 4566 videos of abduction cases hand gestures. These videos of hand gestures have been cropped from custom-recorded videos by volunteers specifically for creating this dataset using a developed algorithm for this purpose. Each cropped video contains 45 frames. Each frame contains one hand represented in 21 keypoint landmarks providing X, Y, and Z coordinates, creating a three-dimensional representation of the hand pose (palm and fingers) in this frame. These custom-videos have been recorded using surveillance cameras mounted at a standard 3-meter height across various environmental conditions, including different lighting scenarios (day/night) and distances up to 17 meters from the surveillance camera. The developping algorithm has been applied to track and crop hand gestures from the custom-videos and save them into new videos as mentioned appove and categorized into two groups 0 and 1. Augmentation techniques have been applied on the new videos to enhance dataset diversity via geometric transformations and visual quality adjustments.Then, the MediaPipe hands framwork had applying on the cropped videos to exctact the 3D hand landmarks. Finally, applying normalization tehcnique on the resulting PKL file of hands landmarks that contain all hands landmarks with their classlables (0 and 1), normalized to range [-1 : 1] to let the researchers deal directly with this dataset in bulding their models of machine learning etc,. Interested researchers could utilize this dataset to develop and train AI models for identifying potential abduction cases through hand gesture recognition in surveillance systems. The data is particularly valuable in scenarios where verbal communication is not feasible. Moreover, the structure of this dataset allows for direct application in training deep learning algorithms or feature extraction for machine learning models, enabling the development of real-time surveillance systems capable of recognizing sign requests for help through hand gestures.