Not from scratch: Explainable deep transfer learning fine-tunning with domain adaptation enables trustworthy COVID-19 prediction
Description
Background and Objective: Medical image analysis can help diagnose Coronavirus Disease 2019 (COVID-19) early and save patient lives before the disease worsens. However, there are various limitations to manual inspection of these medical images, such as dependence on physician experience and subjectivity of assessment. To facilitate the rapid and accurate diagnosis of disease, computer-aided diagnostic systems based on deep learning methods, typically convolutional neural networks (CNN) can be used. However, neural networks are usually black-box models that do not provide a clear insight into their prediction outcomes. Methods: Here, we proposed a framework called explainable deep transfer learning for medical image classification (XDTLMI-Net) that uses four CNNs proficient in handling image data, including GoogLeNet, ResNet18, ResNet50 and ResNet101. This framework uses existing medical domain knowledge to guide transfer learning with COVID-19 CT scan images and CXR images. Results: XDTLMI-Net performed three tasks of medical image classification of COVID-19 on three benchmark datasets: COVID-19 CT, SARS-COV-2 CT and COVID-19 CXR. It achieved an average classification accuracy of 0.9897, 0.9752 and 0.9397, and an average classification F1-score of 0.9898, 0.9741 and 0.9394, respectively. Moreover, we employed the Shaply Additive exPlanations (SHAP) and gradient-weighted class activation mapping (Grad-CAM) to interpret the COVID-19 predictions and help understand the predictive models’ decision-making process. Conclusions: A general end-to-end framework called XDTLMI-Net based on CNN and TL was developed, which works on small datasets of medical images, which does not require any segmentation or image preprocessing procedures. XDTLMI-Net outperformed on three datasets in fine-tuning course and gave reasonable importance to each input COVID-19 image, showing its potential for application in different clinical scenarios.