NCTB-SchoolText: A Curriculum-Aligned Bangla/English Text Corpus for Classes 1–10

Description

NCTB-SchoolText is a passage-level text corpus extracted from the official general-curriculum textbooks published by the National Curriculum and Textbook Board (NCTB) of Bangladesh, covering Classes 1 through 10. The dataset contains 58,872 passage chunks drawn from 1,535 chapters across 34 distinct subjects and nine grade levels (Classes 1–8 individually, and Classes 9–10 combined as the SSC book set), stored in a uniform JSON Lines (JSONL) schema. Each record reports the grade, subject, chapter number, chapter title, a deterministic chunk identifier, and the cleaned passage text, allowing any chunk to betraced back to its exact position in the curriculum. The source materials are scanned, image-based NCTB textbook PDFs. Because direct text extraction from these files is unreliable, the corpus was builtusing optical character recognition (Tesseract, via pytesseract) with Bengali, English, or Arabic language models as appropriate, followed by chapter mapping from per-book configuration files and language-aware cleaning to remove scanning artefacts (headers, footers, page numbers) while preserving punctuation, Bangla numerals, and question/list structure. Text isthen segmented into short, pedagogically coherent chunks (median length 351characters) that keep activities, questions, and expository passages intact rather than splitting them mid-unit. The corpus is bilingual (Bangla and English) and additionally contains Arabic-script content in the Arabic and Islamic-studies books, with Sanskrit and Pali rendered in Bengali script. It is intended as a machine-readable, curriculum-aligned foundation for educational NLP applications such as retrieval-augmented tutoring systems, curriculum-grounded question answering, comprehension-set generation, and low-resource (Bangla) language modelling research. Only processed text chunks and chapter-mapping configuration files are released; raw NCTB PDF scans are not redistributed for copyright reasons.

Steps to reproduce

1. Obtain source PDFs. Official NCTB textbook PDFs for the desired class(es)/subject(s) must be obtained directly from NCTB's own distribution channels, since they are not redistributed with this dataset for copyright reasons. Place each book's PDF inside its class folder, for example classSix/Class6_Bangla.pdf, matching the filename referenced in that class's book catalogue script, classN_books.py. 2. Install dependencies. Python 3 with pdf2image, pytesseract, and Pillow; the Tesseract OCR engine (CLI) with the Bengali (ben), English (eng), and Arabic (ara) language packs installed; and the Poppler utilities, which pdf2image requires to rasterize PDF pages to images. 3. Classes 1 to 5: their tables of contents are image-based and not reliably OCR-able, so chapter boundaries were hand-transcribed from each book's printed table of contents into a BOOKS list (page number, title, and a page offset) inside build_primary.py for Classes 1-2, or the matching build_class<N>.py script for Classes 3-5. Running that script against the source PDF regenerates chapters_config_<subject>.json and processed_chapters_<subject>/*.jsonl for that class. 4. Classes 6 to 10 are machine-readable once OCR'd, so run the class-agnostic driver, in order, for each class N (6, 7, 8, or 9_10): run_class.py N ocr, which OCRs every book listed in that class's classN_books.py catalogue, page by page, into a local cache. run_class.py N process, which for each OCR-complete book auto-detects chapter boundaries using pattern heuristics for numbered, ordinal, section-opener, and lesson layouts, cleans the OCR text, segments it into chunks, and writes chapters_config_<subject>.json and processed_chapters_<subject>/*.jsonl. run_class.py N verify, which validates the six-field schema, chapter and page-range integrity, and chunk_id uniqueness for every subject in that class. A single book can also be reprocessed directly with process_book.py subject pdf_filename, optionally with an english flag and a book name, provided its OCR cache already exists. 5. Determinism. Each chunk_id is generated as class-subject-chapter_no-chunk_index, so re-running the pipeline on the same source PDFs and configuration files reproduces byte-identical chunk text and identifiers.

Institutions

• Leading University

  Sylhet Division, Sylhet

Categories

Licence