Text Corpus and N-gram files for Kurdish Language
Description
The text file contains over 17 million Kurdish Sorani texts. The Kurdish text corpus was collected from different sources such as websites, PDF books, etc. Later, all the texts are unified with Kurdish Unicode font. Then, the corpus pre-processed to omit non-Kurdish words, numbers, symbols, etc. Besides, N-gram/Five gram files for Kurdish language is stored in Excel files. Depending on the Kurdish text corpus, the N-gram files are created. Due to they are utilized to design next word prediction system for Kurdish language. In the system, Stupid BackOff SBO algorithm hybridized with N-gram, also SBO and Improved Fitness Dependent Optimizer IFDO are hybridized to find the best values for backing off parameter Lambda in SBO to BackOff from quad gram to trigram, trigram to bigram, and bigram to unigram. The system Accuracy was %96.3.