Performance Validation of Deep Learning-based Models for the Quantification of COVID-19 Pneumonia and Its Application to Development of Machine Learning Model for Differentiating COVID-19 Cases from ILD and Normal Cases Using Chest CT
- Nov. 2020
- by Beomhee Park et. al.
To evaluate the performance of two machine learning models; (i) convolutional neural network(CNN) model for the quantification of lesions related to COVID-19 pneumonia, (ii) random forest model for differentiating COVID-19 cases from interstitial lung disease(ILD) and normal cases based on the quantification result
METHOD AND MATERIALS
The CNN-based quantification model was originally developed in the earlier study to quantify ILD-related patterns and modified to predict only COVID-19 pneumonia related patterns such as ground-glass opacity(GGO), reticular patterns, and consolidation. A random forest model was trained to classify COVID-19 cases from non-COVID-19 cases based on the features calculated from the quantification result. The features used were representative CT findings of COVID-19 pneumonia; relative volume of each patterns to whole lung volume, relative volume of each pattern to all lesion volume, number of affected lobes, and binary indicator of lesion distribution(bilateral, posterior, peripheral, basal). The training set consists of 57 COVID-19 confirmed cases as positive samples and combination of 96 ILD and 496 NLST cases as negative samples. The performance of quantification and classification was evaluated with separate samples consisting of 20 COVID-19, 95 ILD, 496 NLST cases.
The Dice similarity coefficient for lesion quantification in COVID-19 confirmed cases was 0.685, which is compatible to the performance of a previously reported model explicitly trained to segment COVID-19 lesions. The performance of model for differentiating COVID-19 cases from NLST cases was 0.989, 1.00 and 0.956 in AUC, sensitivity and specificity, while performance of differentiating COVID-19 from ILD was lower with 0.906, 1.00 and 0.674, respectively. The most significant features for differentiating COVID-19 from ILD was the relative volume of GGO and basal distribution of lesions.
The CNN-based model developed for quantifying ILD patterns showed moderate performance for quantifying COVID-19 pneumonia lesions. While the sensitivity of detecting COVID-19 cases is high, differentiating COVID-19 pneumonia from ILD was relatively difficult due to the similarity of CT findings in two diseases.