Bangla Lip-to-Text Sentence Level Dataset
Description
The dataset is created as follows: Step 1: Original Bangla videos were collected from YouTube Bangladesh National Television Debates and Talk Shows. Step 2: Original videos are cut into 3-second duration mini videos. Step 3: Face area from each frame of these mini videos is segmented using MediaPipe FaceMesh. Step 4: Only those videos were chosen in which all of the frames contain exactly one face. Step 5: Audios are cut and embedded in the chosen videos (folder: Face_Only_Videos_with_Audio). Step 6: Bangla audio-to-text transcriptions of the chosen videos were generated using the Wav2Vec2 foundation model pre-trained on the Mozilla 300m Bengali Commonvoice dataset (folder: First_Transcriptions). Step 7: Videos with no transcription texts were filtered out. Step 8: A simplified transliteration for each of the translations was created (folder: Transliteration_Informal).
Files
Institutions
- Khulna University