Kiswahili Hate speech dataset

Description

Kiswahili Hate Speech Dataset (KHS-2026) Overview KHS-2026 is a monolingual Kiswahili dataset created for hate speech detection in low-resource African languages. It contains 6,451 annotated social media posts/comments collected from public sources in East Africa, primarily Kenya. Size & Classes Total: 6,451 items Neutral: ~65% (≈4,285) Offensive: ~25% (≈1,565) Hate Speech: ~10% (≈601) The severe imbalance reflects real-world social media, where most content is neutral or mildly critical, while explicit hate is rare but harmful. Sources Public posts/comments from: Twitter (now X) Facebook YouTube Focus: Sensitive topics including politics, protests, gender, ethnicity, and religion (mostly Kenyan discourse). Collection & Preprocessing Raw collection: ~6,539 items (Twitter API, Facebook CrowdTangle, YouTube API + manual) Preprocessing: Conservative cleaning Removed duplicates Normalized whitespace & UTF-8 Filtered broken URLs, excess punctuation, irrelevant metadata Kept emojis, hashtags, mentions for context Minimal spelling fixes; no stemming/lemmatization Final size after cleaning: 6,451 items Annotation Annotators: 3 native/fluent Kiswahili speakers (linguistics/comms background) Training: 2-day session + pilot on 500 samples Scheme: 3 primary labels – Neutral / Offensive / Hate Speech Inter-annotator agreement: Free-marginal Randolph’s Kappa = 0.72 (fair/good) Disagreements: Resolved via discussion + majority vote Validation Baseline models (TF-IDF + 75/25 split): Overall accuracy: ~69% Logistic Regression: Neutral F1 ≈ 0.81; Offensive/Hate F1 ≈ 0.41 each SVM: Strong precision but very low recall on minorities (F1 0.13–0.22) Poor minority-class performance due to imbalance + subtle expressions (sarcasm, metaphors, cultural nuance). Purpose & Release Provides a culturally grounded resource for Kiswahili NLP, complements AfriHate, and supports content moderation in East/Central Africa. Planned release: Public under CC BY-NC 4.0 with datasheet (following Gebru et al., 2018) detailing sources, biases, limitations (dialect variation, code-mixing risks, subtle hostility). Key insight The dataset shows reliable neutral annotation but highlights major challenges in detecting nuanced hate/offensive content in African languages – a call for better imbalance handling and culturally informed models.

Steps to reproduce

Reproducing the Kiswahili Hate Speech Dataset (KHS-2026) (target: ~6,451 annotated monolingual Kiswahili posts), adapted for 2026 constraints (e.g., API changes, CrowdTangle shutdown). Define Scope & Ethics Focus: Public Kiswahili posts/comments from politics, gender, ethnicity, religion (mostly Kenyan contexts). Define 3-class scheme: Neutral / Offensive / Hate Speech (with guidelines for sarcasm/ambiguity). Secure ethics approval; comply with ToS, anonymize (no PII), follow UNESCO AI Ethics (2021) & Kenya Data Protection Act. Data Collection (~6,500–7,000 raw items) X/Twitter: Use X API v2 (academic access if possible); query Kiswahili keywords/hashtags (e.g., "Ruto", "maandamano", "polisi") + lang:sw filter. Facebook: Use Meta Content Library (research application) or compliant manual/API alternatives. YouTube: YouTube Data API v3 for public Kiswahili video comments (filter by language/keywords). Supplement manually if needed. Respect rate limits & public-only data. Preprocessing (conservative) Remove duplicates; normalize whitespace/UTF-8. Filter noise (broken URLs, excess punctuation, irrelevant metadata). Retain emojis, hashtags, mentions for context. Minimal typo fixes; no stemming/lemmatization. Output: ~6,451 cleaned items. Annotator Setup Recruit 3+ native Kiswahili speakers (linguistics/comms background). 2-day training: definitions (psych/legal/UN), tools (e.g., LabelStudio), ambiguity handling. Pilot on ~500 samples; refine guidelines. Annotation Independent labeling (Neutral/Offensive/Hate). Compute free-marginal Randolph’s Kappa (~0.7+ target). Resolve via discussion + majority vote. Optional: add intensity/target tags. Quality & Validation Random audits & feedback loops. Baseline ML: TF-IDF + Logistic Regression/SVM (75/25 split). Expect ~69% accuracy, neutral F1 ~0.81, minority F1 ~0.41 (due to imbalance). Finalize & Release Bundle: texts + labels (+ explanations). Create datasheet (Gebru et al., 2018): sources, limits (imbalance, dialects, code-mixing), biases. License: CC BY-NC 4.0.

Institutions

• KCA University

  Nairobi County, Nairobi

Categories

Licence