Introduction
Pulmonary nodules are common radiological findings observed in the day-to-day practice of a radiologist. They are reported in 1.6 million patients per year in the U.S. [1]. 19-26% of the lung nodules that are apparent on the chest radiographs are overlooked at their first readings [2,3]. A missed lung nodule is a subject of concern among radiologists and is a serious medicolegal issue. The appropriate and timely detection of pulmonary nodules can lead to early treatment of the pathology while minimizing the testing required for benign nodules.[1]
We identified that an artificial intelligence (AI)-powered approach capable of assisting in detecting pulmonary nodules on chest radiographs could greatly augment a radiologist’s ability to detect such nodules, hence improving healthcare outcomes. The use of chest radiography in diagnostic practice is often the first-line imaging investigation for the diagnosis of suspected respiratory disorders. The use of Chest X-Rays as a first-line investigation is attributed to its ease of procedure, wide-scale availability, accessibility, and lower radiation exposure as compared to chest CT scans. These advantages are also the reason that this mode of investigation is more prevalent in peripheral healthcare settings, ensuring its wide-scale acceptability. Chest X-rays, however, are also the most common cause of missed lesion identification, accounting for nearly 90% of all missed lung cancers [4]. Therefore, using an AI model for identifying lung lesions from chest radiographs will ensure better outcomes for patients.
AI can provide virtual support to all human readers by functioning as concurrent readers. A deep-learning-based AI model can improve the performance of reviewing radiologists and save their time by screening and identifying each radiograph, thereby allowing radiologists to do more targeted and speedy reporting [5].
Methodology
We developed a deep learning model called DxNodule AI Screen to detect pulmonary nodules on chest radiographs and compared its performance with expert readers having varying degrees of experience in diagnosing the disease. The study included 308 frontal chest radiographs belonging to 308 adult patients from a tertiary care hospital. The consensus of three board-certified radiologists was utilized to establish the ground truth for these radiographs. The deep learning model architecture wasan ensemble of two Feature Pyramid Networks(FPN1 and FPN2), each having an Xception encoder.
To assess the performance of the DxNodule AI Screen, and to establish its clinical utility, it was necessary to evaluate its performance against the performance of multiple readers. It was also important to evaluate the difference in the performance of the readers when the model’s output was provided as an aid to them. The radiologists who determined the ground truth did not participate as readers. We then compared the performance of 11 readers without and with the aid of the DxNodule AI Screen.
Results
The findings were considered suggestive of the pulmonary nodule(s) if they had a size between 5-30 mm. These nodules were seen individually or in a cluster in the right and/or the left lung field. These were calcified/non-calcified and were distributed in the upper, mid, or lower zones on the right or left side of the lung field.
The standalone performance of the model in detecting pulmonary nodules on chest radiographs was noteworthy. DxNodule AI Screen had an accuracy of 83%, a sensitivity of 78%, a specificity of 88%, and an AUC of 90.5%. All 11 readers assessed radiographs without the aid of DxNodule AI Screen and then with the aid of DxNodule AI Screen after a washout period of a month.The mean performance of the readers improved with DxNodule AI Screen-aided interpretation. There was a decrease in the average number of false-negative (31.58 to 24.64) and false positive (26.42 to 22.18) identifications and an increase in the average number of true negative (178.58 to 182.82) and true positive (71.42 to 78.36) identifications compared with unaided interpretation.
With the DxNodule AI Screen-aided interpretation, the specificity, balanced accuracy, PPV, and NPV across the 11 readers improved with statistical significance (p<0.05) when compared to the unaided interpretation. (As seen in Table 1).
Table 1: Sensitivity, Specificity, Accuracy, NPV, and PPV of unaided and DxNodule AI Screen-aided interpretation modes for pulmonary nodule detection
The mean sensitivity of the readers improved from 69% in the unaided session to 76% in the aided session. Similarly, the mean specificity of the readers improved from 87% in the unaided session to 89% in the aided session. These results are important as they may help reduce the number of unnecessary follow-up procedures, which may pose a radiation risk to the patient. The results indicate that DxNodule AI Screen can function as a reliable concurrent reader and increase the accuracy of the reports and augment the performance of imaging experts.
There was a significant difference in the performance of the readers aided by the DxNodule AI Screen and those not aided by the DxNodule AI Screen. The diagnostic performance of the unaided readers was compared to aided readers and represented as the AUROC curve. The average AUC of the readers improved from 79.8% to 84.6% when aided by DxNodule AI Screen (p-value = 0.013). Standalone DxNodule AI Screen achieved an AUC of 90.5% in identifying pulmonary nodules in the test dataset (Fig 1).
Figure 1: AUROC curves depicting the performance of standalone DxNodule AI Screen(red), AI-unaided readers (green), and AI-aided readers (blue)
Examples of clear improvement seen in lung nodule detection rates when DxNodule AI Screen supplements the readers
DxNodule AI Screen assisted the radiologists in identifying nodules that were otherwise missed due to a wide variety of factors like overlying ribs or scapula shadow, the small size of nodules, etc. In Fig 2, a rib overshadowed the nodule which led to low rates of detection but with the help of the DxNodule AI Screen, the detection rate significantly increased.
DxNodule not only helped readers identify radiographs with nodules, but it also helped readers locate them more accurately as seen in Fig 3, wherein the readers were able to detect the lung nodules at a higher rate and greater accuracy with the help of DxNodule AI Screen.
Figure 2: Lung nodule overshadowed by the posterior end of the seventh rib in the left lung. Only 1 out of 11 readers detected this without DxNodule AI Screen. With DxNodule AI Screen, 8 were able to detect.
Figure 3: (a) Original chest radiograph with two nodules in the left lung. (b) Ground truth masks indicate the location of nodules in the chest radiograph. (c) Location of nodules as predicted by the DxNodule AI Screen. (d) Without the A.I. assistance, only one reader properly identified both nodules, while seven radiologists identified only one nodule, two readers identified nodules at more than two places (false positive), and one reader marked the radiograph as negative (false negative). (e) When aided by A.I., four readers could identify both nodules (true positive), six readers marked one nodule as positive, and one radiologist marked the radiograph as negative. There were no false-positive findings.
The above examples and prior results suggest that AI can be a game-changer for pulmonary nodule assessment on chest radiographs as it can assist the experts and augment the imaging process. DxNodule AI Screen can be used as a complementary reader for the detection of pulmonary nodules, making it a suitable candidate for inclusion in the diagnostic arsenal of a radiologist.
References
The full article is available at:https://www.medrxiv.org/content/10.1101/2022.06.21.22276691v2