Mexican Sign Language's Dactylology and Ten First Numbers - Extracted Features and Models
Description
The following folders have the extracted landmarks coordinates by Mediapipe Holistic that were obtained from videos of Mexican Sign Language's Dactylology and Ten First Numbers to get static (image) and continuous (video) signs respectively. Additionally, the static (SVM, GBL) and continuous (GRU, LSTM) trained models were saved under the folders' names static_data and dynamic_data accordingly. LSTM's final model was saved under the name: "dynamic_experiment_lstm_v4.h5" GRU's final model was saved under the name: "dynamic_experiment_lstm_v7.h5" Gradiant Boost Light's final model was saved under the name: "models_histgrad_v2" SVM's final model was saved under the name: "models_svm_v2"
Files
Steps to reproduce
1) Save recordings of sign language Dactylology and Ten First Numbers. 2) Edit the recording according to the length of the sign performance for continuous signs. For static signs, you can take a screenshot. 3) Write the name of the sign in the file name. 4) Organize the folders so that the files can be ordered by person, then by the number of the cycle, and lastly, by the hand that was used. 5) Use MediaPipe Holistic and subtract the landmarks that are only used for hands (42 points in total for both hands). Every landmark's information will give you its coordinate (X, Y, Z). 6) Save the landmark's coordinates accordingly to the sign that is being performed. 7) For static signs, use Support Vector Machine (SVM) and Gradiant Boost Light (GBL) to train the models, 80% of data use it for training, 10% for testing, and 10% for validation. Use Grid Search during the training to get the best params, and use the library scikit-learn for the models. 8) For continuous signs, use Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) to train the models, 80% of data use it for training, 10% for testing, and 10% for validation. Use Tensorflow as the library for the models, and apply Tensorflowboard to get the logs of statistics. 9) Save the models according to your experiments' results and versions.