Methodological Appraisal and Credibility Assessment checklist (MACA)
Description
This dataset accompanies the Methodological Appraisal and Credibility Assessment checklist (MACA), developed to evaluate the methodological quality of studies applying computational and statistical methods for composite indicator construction. The repository includes three reproducible components: MACA_ICC-KAPPA: Scripts and results for inter-rater reliability analysis (Cohen’s Kappa, Gwet’s AC1, PABAK, and ICC) of the MACA checklist. MACA_FinalScore: Script and output for calculating the averaged and total scores of the final 17-item fused version of MACA, based on two independent evaluators. MACA_heatmap: Python script and visualization for the heatmap summarizing methodological quality across studies. All scripts are written in Python and include example input and output files for transparency and reproducibility. The dataset supports the umbrella review on methodological quality assessment in computational and statistical methods applied to public health and composite indicators.
Files
Steps to reproduce
1. Download the file *Consolidated_Dataset_MACA.xlsx* and the Python scripts provided in each folder. 2. Open the corresponding Jupyter Notebook (.ipynb) or Python script (.py) in any Python 3.9+ environment (e.g., Anaconda, JupyterLab, or VS Code). 3. Install the required Python libraries: pandas, numpy, pingouin, scikit-learn, and openpyxl. 4. For inter-rater reliability (ICC-KAPPA analysis): - Run the script *MACA_ICC-KAPPA.ipynb*. - It calculates Cohen’s Kappa, Gwet’s AC1, PABAK, and ICC for all items, dimensions, and the global MACA checklist. 5. For the final MACA score: - Run *MACA_FinalScore.ipynb* to compute averaged item scores across evaluators and the total MACA_Score_Final. 6. For the heatmap visualization: - Run *MACA_heatmap_generator.ipynb* to reproduce the *MACA_heatmap.png* output. 7. The outputs (*IRR_results-36.xlsx*, *IRR_results-17.xlsx*, *MACA17_MeanScores.xlsx*) will be automatically generated in the same directory. 8. All scripts are self-contained and can be executed without modification if the input Excel files are located in the same path.