Moroccan News Dataset and Source Code for Arabic and French Topic Detection
Description
This dataset contains news articles collected from 37 Moroccan press sources (21 Arabic and 16 French) via RSS feeds over a period of 8 months, from June 14, 2025 to February 14, 2026. The dataset comprises 729 Arabic files and 729 French files, corresponding to one file per system update (3 updates per day). After deduplication, the dataset contains 115145 unique Arabic articles and 55239 French articles. Each file is stored in Python Pickle format and contains a list of dictionaries, where each dictionary represents a news article with the following fields : title, content, source, url, and date. A Python script for reading and preprocessing the data is provided alongside the dataset. This dataset was created to support a study on automated news topic detection. Along side the dataset, we provide the source code of the of the proposed monitoring pipeline : data collection, preprocessing, transformer-based (BERT/AraBERT) contextual embedding, topic detection using Graph Community Analysis, and treemap and graph visualizations. To simplify deployment, the code comes with transformer models and NLP tools, allowing for immediate use on the provided data. Detailed instructions on how to set up the environment and run the topic detection pipeline are provided in the README file.
Files
Steps to reproduce
News articles were automatically collected from 37 Moroccan press sources (21 Arabic and 16 French) via RSS feeds using a Python script deployed on GitHub Actions. The system runs automatically 3 times per day, collecting at each update the articles published in the last 24 hours. Each update is stored as a separate Pickle file containing a list of dictionaries, where each dictionary represents one article with the following fields : title, content, source, url, and date. The collection period spans 8 months, from June 14, 2025 to February 14, 2026, resulting in 729 files per language. After deduplication based on article title, the dataset contains 115,145 unique Arabic articles and 55,239 French articles. The full source code of the porposed topic detection system, is provided alongside the dataset. Detailed instructions on how to set up the environment and run the topic detection pipeline are provided in the README file.