Adyan: A High-Quality Automated NER Dataset for Sorani Kurdish
Description
The Named Entity Recognition Dataset for Kurdish Sorani (Adian) is an automated annotated resource created to support NLP studies for Kurdish Sorani, a low-resource language. By providing a large, structured dataset that trains and evaluates Kurdish language models, it addresses the lack of annotated corpora for nominal entity identification (NER). The dataset of more than 2,300 news articles published in 2024 by trusted Kurdish media sources such as Rudaw, NRT,Avanews and Kurdsatnews covers six main domains: politics, economy, sports, culture, interviews, and technology. It contains 654,404 tokens across 22,801 strings. A predefined dictionary of 12,030 named entities was used for annotation, covering 15 entity types. Annotation was performed automatically using a dictionary-based approach and the BIO tagging scheme. Preprocessing steps such as text cleaning, normalization, and duplicate removal were implemented using Python scripts to improve consistency and quality. Adyan is publicly available for academic and research use. Its scale, domain coverage and automated methodology make it a valuable resource not only for NER but also for sentiment analysis, tool translation and other Kurdish NLP tasks. The dataset makes a significant contribution to the advancement of language technology for underrepresented languages.