BanglaNER-Corp: A Novel Reference Corpus for Bangla Named Entity Recognition
Description
This repository contains a Bangla Named Entity Recognition (NER) dataset along with the source code used for data preprocessing. The dataset is a hybrid Bangla Named Entity Recognition (NER) corpus designed to support research on low-resource natural language processing. It combines translated and native Bangla text to provide a diverse and high-quality benchmark for training and evaluating NER models. Approximately 70% of the corpus was created by translating the English CoNLL-2003 dataset into Bangla using neural machine translation models (BanglaT5, NLLB, and Google Translate), followed by manual verification and correction to preserve semantic meaning and linguistic fluency. The original BIO entity annotations for Person (PER), Organization (ORG), Location (LOC), and Miscellaneous (MISC) were retained and carefully realigned with the translated text. The remaining 30% consists of manually collected and annotated Bangla news sentences spanning multiple domains, including politics, sports, international affairs, culture, weather, mythology, religion, science, and geography. The complete corpus contains 6,162 annotated sentences and 69,193 tokens, with a vocabulary of 14,349 unique tokens. Entity annotations include 4,609 PER, 2,904 ORG, 4,678 LOC, and 3,172 MISC tokens, alongside 53,830 non-entity (O) tokens. All annotations follow the BIO tagging scheme and are provided in CoNLL format, making the dataset directly compatible with modern NLP frameworks such as Hugging Face Transformers, PyTorch, TensorFlow, and spaCy. In addition to the dataset, this repository includes the complete source code for data preprocessing. This dataset is intended for training, evaluation, benchmarking, and comparative research on Bangla NER systems, particularly transformer-based and hybrid sequence-labeling models. It also serves as a valuable resource for downstream Bangla NLP applications, including information extraction, question answering, and knowledge graph construction.
Files
Institutions
- Khulna University of Engineering and TechnologyKhulna Division, Khulna