Multi-Class Chronic Disease Data Warehouse (healthcare)

Description

This dataset represents an integrated medical data warehouse developed to support multi-class chronic disease prediction. It combines three publicly available healthcare datasets—diabetes, heart disease, and hypertension—sourced from Kaggle and unified using a Medallion Architecture (Bronze, Silver, Gold) implemented in Microsoft SQL Server. The final Gold-layer dataset contains 280,985 patient records and 38 features, with no missing values. Each record corresponds to a patient and includes both a binary classification (Normal/Abnormal) and an 8-class sublabel representing disease combinations, enabling advanced co-morbidity analysis and predictive modeling . The dataset is structured as a denormalized flat table derived from a star schema and captures comprehensive patient profiles across five domains: demographic attributes (e.g., age, gender), anthropometric measures (e.g., BMI), lifestyle indicators (e.g., smoking, physical activity, stress), clinical measurements (e.g., glucose, HbA1c, cholesterol, blood pressure), and disease indicators. Features include both categorical and continuous variables, such as normalized age, lipid profiles, inflammatory markers (CRP), and cardiovascular metrics. Disease representation is encoded through binary flags and a composite categorical sublabel capturing all possible combinations of diabetes (DI), heart disease (HT), and hypertension (HY). The dataset was designed to address limitations in single-disease modeling by enabling simultaneous prediction of multiple chronic conditions. It supports the study of shared risk factors and cross-disease interactions, providing a unified feature space for machine learning applications. This facilitates the development of clinical decision-support systems capable of early detection, risk stratification, and holistic patient assessment. Provided as a UTF-8 encoded CSV file, the dataset is compatible with major analytical platforms such as Python, R, and SQL tools. Ethical considerations are addressed through full anonymization of all source data, with no personally identifiable information included. Potential applications include multi-class classification, co-morbidity analysis, feature importance studies, and benchmarking of machine learning models. It also serves as an educational resource for data warehousing and healthcare analytics. Keywords associated with the dataset include chronic disease classification, Medallion Architecture, clinical decision support, and machine learning-based healthcare analytics.

Steps to reproduce

The production of the Gold-layer analytical dataset represents the final stage of a refined data engineering pipeline, transitioning from raw source ingestion to a high-utility clinical feature space. This process, implemented within Microsoft SQL Server, follows a structured methodology to ensure data integrity and clinical relevance. 1. Architectural Foundation: Medallion Framework The pipeline utilizes a Medallion Architecture to incrementally refine data: • Bronze Layer: Ingestion of raw, independent healthcare datasets (Diabetes, Heart Disease, and Hypertension) sourced from Kaggle. • Silver Layer: Cleansing, standardization, and initial transformation. This includes dropping non-essential attributes, such as geographical location data, to maintain strict anonymization protocols. • Gold Layer: The final transformation into a denormalized star schema optimized for multi-class chronic disease analysis. 2. Schema Harmonization and Data Integration The core of the Gold-layer production involves consolidating heterogeneous schemas into a unified clinical framework: • Consolidation: Integrating 280,985 patient records from three complementary sources. • Standardization: Mapping disparate categorical values (e.g., smoking status, physical activity) into a standardized format across all records. • Feature Engineering: Deriving 38 distinct features across demographic, anthropometric, lifestyle, and clinical domains. 3. Clinical Data Enrichment and Imputation To ensure a robust "Gold" standard, specific logic is applied to handle source-specific variances: • Logical Imputation: Indicators for diseases not present in a specific source (e.g., hypertension flags for non-diabetes records) are strategically imputed to maintain a complete 3-bit disease string (DI, HT, HY). • Feature Normalization: Applying Min-Max normalization to continuous variables like age (range [0.00, 1.00]) and deriving binned categories such as age_level or bmi_level. • Clinical Flagging: Generating binary flags for clinical thresholds, such as low_hdl_cholesterol (HDL < 40) or high_ldl_cholesterol (LDL ≥ 130). 4. Target Labeling and Final Export The final step defines the analytical targets required for multi-class and binary research: • Multi-Class Labeling: Creating an 8-class sublabel (e.g., DI_HT_HY, N) to represent complex co-morbidity patterns. • Binary Synthesis: Determining the final health screening target—Normal (106,884) vs. Abnormal (174,101)—based on the presence of any tracked condition. • Denormalization: Flattening the Gold-layer star schema into a single analytical table (patient_flat.csv) for seamless interoperability with research tools like Python, R, and SQL.

Institutions

• New Mansoura University

  Dakahlia, Al Mansurah
• Mansoura University

  Dakahlia, Al Mansurah

Categories

Related Links

Licence