(KNQAD): Kurdish News Question answering Dataset
Description
Question answering (QA) is the field of information retrieval (IR) aimed at answering questions from paragraphs in natural language processing (NLP). Essentially, IR is a technique to retrieve and rank documents based on keywords, while in the QA system, answers to questions are retrieved based on the paragraph's content. The Kurdish language belongs to the Indo-European family spoken by 30-40 million people worldwide. Almost all Kurdish people speak Sorani and Kurmanji dialects. In this dataset, the Sorani dialect is used as the first attempt to collect and create a Kurdish News Question-Answering Dataset (KNQAD). The texts are collected from numerous Kurdish news websites covering various fields such as religion, social issues, art, health, economy, politics, sports, and more. In this project, 15,002 question-answer pairs are created manually from 15,002 paragraphs. Three preprocessing steps are implemented on the raw text paragraphs: stemming, removing stop words, and removing special characters.