Risk of IT project failure

Published: 3 November 2026| Version 1 | DOI: 10.17632/2ghss2d3cb.1
Contributors:
,

Description

The primary objective of this study is to assess the probability that a project will not be executed in accordance with its planned parameters. To achieve this, it is necessary to analyze the key characteristics of IT projects and their impact on timely execution. Project non-fulfillment is defined as a situation involving a significant deviation from the plan in one or more of the following areas: exceeding the approved budget; missing planned deadlines; failure to complete the planned scope of work; a decline in the quality of deliverables; inability to meet the client’s key requirements. The risk of non-fulfillment refers to the potential occurrence of a problem, which must be evaluated before it actually arises. The machine learning model is intended to predict this risk based on information available at the time of assessment, rather than on outcomes that only become known after the project is completed. The created dataset contains 5,000 records, each describing an individual IT project. It consists of a mix of real and synthetic data. For each project, characteristics that may influence the success of its implementation have been defined. The target variable, `target_project_risk_class`, has three possible values—Low, Medium, and High—corresponding to the respective risk levels. The dataset features are divided into several functional groups. Project characteristics describe the overall complexity and scale of a project and include: project_type; industry; methodology (Agile, Waterfall, Hybrid); complexity; priority; team_size. Human and resource factors allow for an assessment of whether the team possesses sufficient resources and competencies to execute planned tasks, specifically: team_experience_years; resource_utilization_pct; expected_team_turnover; stakeholder_engagement. Planning and management factors allow for an evaluation of planning quality, requirement stability, and the complexity of coordinating work: planned_budget_uah; planned_duration_months; planned_effort_hours (in hours); requirements_volatility_pct; dependency_count; vendor_count; risk_register_count; mitigation_maturity. Technological and organizational factors include the following: legacy_integration; data_quality_maturity; security_requirements; governance_maturity. The dataset is intended for training a machine learning model. Once the model is trained, it is important not only to obtain a prediction but also to determine which specific factors have the greatest impact on the formation of risk. To achieve this, one can use: • Feature Importance for assessing the importance of features in a Random Forest; • SHAP for explaining the contribution of individual factors to the prediction; • Partial Dependence Plot for analyzing how the prediction depends on a specific feature.

Files

Steps to reproduce

Before training the model, it is necessary to separate the target variable `target_project_risk_class`. All other valid project characteristics form the feature matrix. This dataset contains fields that describe the actual outcomes of a project: schedule_delay_days; actual_duration_days; budget_overrun_pct; actual_budget_uah; deliverable_completion_pct; quality_score_1_5. These cannot be used for risk prediction at the start of a project. They only become known during or after the execution of the work and may, in effect, reveal the very answer that the model is intended to predict. It is also necessary to encode categorical variables. For example, the `methodology` variable contains the values ​​Agile, Waterfall, and Hybrid. To be used in most machine learning algorithms, these categories must be converted into a numerical representation. One-Hot Encoding can be applied. For algorithms sensitive to feature scale (logistic regression, SVM, KNN), it is advisable to apply standardization or normalization. For Random Forest and other tree-based algorithms, scaling is generally not required. Before conducting training sessions, the dataset needs to be split into training (80%) and test (20%) sets. In the provided dataset, the target class is distributed unevenly (low-risk projects account for approximately 70%, medium-risk projects for about 27%, and high-risk projects for 3%). While this generally reflects real-world conditions, relying solely on Accuracy to evaluate model performance is not advisable in this context. Since the "high-risk" class represents only 3.02% of the data, a model could achieve high overall accuracy while failing to identify the majority of high-risk projects. If Accuracy shows the overall proportion of correct predictions, then a number of the following metrics will provide a broader picture of model quality. Thus, Precision will show the proportion of truly high-risk projects among those that the model classified as high-risk, Recall will calculate the proportion of detected high-risk projects among all such projects, F1-score will characterize the balance between Precision and Recall, Macro-F1 will provide the average F1 value for all classes with the same class weight, and Confusion Matrix will show the distribution of correct and false classifications. It should be emphasized that for the task of IT project management, Recall for the high class is especially important: if the model does not detect a project with a high risk, the management may not take timely measures. In addition, to overcome the imbalance, you can apply class_weight='balanced'; setting classification thresholds; oversampling of the training sample (for example SMOTE); comparison of Macro-F1 and Recall class high. Balancing methods should only be applied to the training sample, not the test sample.

Categories

Management, Risk Analysis, Neural Network

Licence