Synthetic Benchmark Dataset and Reproducibility Artifacts for Pre-Execution Validation of AI-Assisted Supplier Selection in ERP Systems
Description
This dataset and reproducibility package supports the research on pre-execution validation of AI-assisted supplier-selection decisions in ERP systems. The repository contains two controlled synthetic benchmark datasets together with the code, experimental results, figures, model artifacts, environment information, and documentation required to reproduce the reported analyses. Experiment 1 is a deterministic controlled benchmark in which unsafe decisions are generated from explicit rule-based conditions. It is intended primarily as a rule-recovery and implementation-verification benchmark rather than as evidence of predictive discovery in a real ERP environment. Experiment 2 introduces probabilistic target generation through a latent risk function and Bernoulli sampling. It provides a less deterministic synthetic setting for evaluating classification performance, threshold sensitivity, calibration, asymmetric false-negative/false-positive costs, and feature-ranking stability. Version 2 extends the reproducibility materials following peer-review revision of the associated manuscript. The update adds evaluation of a hybrid validation architecture combining deterministic hard-constraint checks with machine-learning-based residual-risk validation, repeated stratified holdout analysis across multiple random seeds, threshold-sensitivity analysis, asymmetric cost analysis, calibration metrics, feature-ranking stability analysis, updated figures and result tables, and expanded reproducibility documentation. The original synthetic benchmark datasets used in Version 1 are preserved without modification. Version 2 therefore extends the analysis and reproducibility evidence while maintaining continuity with the original experimental data. The materials are intended to support reproducibility, independent verification, methodological extension, and future research on reliable pre-execution validation of AI-assisted ERP procurement decisions.
Files
Steps to reproduce
1. Download and extract the reproducibility package. 2. Use Python 3.13.5 or a compatible Python environment. The validated environment used NumPy 2.3.5, pandas 2.2.3, scikit-learn 1.8.0, XGBoost 3.1.3, matplotlib 3.10.8, and joblib 1.5.3. The exact environment information is also provided in the Environment folder. 3. Install the required Python dependencies using the supplied environment or requirements specification. 4. Preserve the provided directory structure. The package contains the Code, Data, Environment, Figures, Models, Results, and supporting documentation required for reproduction. 5. Run the main analysis workflow from the Code folder. The supplied code loads the archived synthetic benchmark datasets and reproduces the deterministic-rule, machine-learning-only, and hybrid rule-plus-machine-learning evaluations. 6. Experiment 1 contains 5,000 synthetic procurement records and is a controlled deterministic rule-recovery and implementation-verification benchmark. 7. Experiment 2 contains 10,000 synthetic procurement records and uses a documented latent risk function and Bernoulli sampling to create a probabilistic synthetic benchmark. 8. The `unsafe_probability` field in Experiment 2 is retained for data-generation provenance and verification only. It is not used as an input feature for model training or evaluation. 9. The machine-learning evaluations include Logistic Regression, Random Forest, and XGBoost. The revised analysis additionally evaluates deterministic hard constraints, machine-learning-only validation, and a hybrid architecture combining hard-rule eligibility checks with machine-learning-based residual-risk validation. 10. Repeated evaluation is performed using stratified 80/20 holdout splits across the documented random seeds. The package also reproduces threshold-sensitivity analysis, asymmetric false-negative/false-positive cost analysis, calibration metrics, and feature-ranking stability analysis. 11. Generated results, figures, model artifacts, and analysis outputs can be compared with the corresponding files supplied in the Results, Figures, and Models folders. 12. The manifest and SHA-256 checksum files can be used to verify package integrity. Refer to the README file for file descriptions, execution guidance, model configurations, and interpretation of the generated outputs. The datasets and outputs are entirely synthetic and contain no real ERP transactions, suppliers, organizations, personal data, or confidential enterprise information.
Institutions
- Cairo UniversityGiza, Giza