Darbest Dataset: Universal Dependencies Treebank for Standard Sorani Kurdish

Description

The Darbest dataset is a Universal Dependencies (UD) treebank dataset for Standard Sorani Kurdish written in the Perso-Arabic script. It contains 69,000 annotated sentences and 1,205,855 tokens collected from nine textual domains. The corpus was collected from seven Kurdish online news websites and supplemented with texts from published books. Before preprocessing, the collected corpus contained 1,250,275 words from 5,627 web pages together with book-based texts and was preprocessed using a Python-based pipeline involving text cleaning, Unicode and punctuation normalization, sentence segmentation, and tokenization. The dataset was developed to provide a large-scale syntactically and morphologically annotated resource for Standard Sorani Kurdish. A separate 100-sentence gold-standard set was manually annotated according to the Universal Dependencies v2 guidelines. Sorani Kurdish linguistic experts supported the selection of sentences representing diverse and linguistically complex structures and reviewed LLM-generated annotations for errors. The gold-standard set was used to construct few-shot prompts for annotating the remaining corpus. The resulting annotations were represented in the standard CoNLL-U format and validated using the official Universal Dependencies validation tool, followed by manual correction and quality review. The released treebank is divided into training, development, and test sets and includes lemmas, Universal Part-of-Speech (UPOS) tags, morphological features, syntactic heads, and dependency relations. The dataset can be used to train, evaluate, and benchmark NLP models for part-of-speech tagging, lemmatization, morphological analysis, dependency parsing, and related computational linguistics tasks. The accompanying repository also contains the separate 100-sentence gold-standard set, plain-text corpus splits, README documentation, and a dataset statistics spreadsheet.

Institutions

• Sulaimani Polytechnic University

  Sulaymaniyah, Sulaymaniyah

Categories

Licence