AI assistance when diagnosing clinically significant prostate cancer (csPCa) on MRI is superior to unassisted readings, according to a study published June 13 in JAMA Network Open.
The finding is from an evaluation involving 61 readers (34 experts and 27 nonexperts) from 53 centers across 17 countries and highlights the potential added value of AI assistance during radiologic assessments, noted lead author Jasper Twilt, a doctoral student at Radboud University Medical Center in Nijmegen, the Netherlands, and colleagues.
“AI assistance was associated with a statistically superior improvement in detecting csPCa, increasing the area under the receiver operating characteristic curve, sensitivity, and specificity compared with unassisted readings,” the group wrote.
Despite widespread adoption of prostate MRI in clinical practice and the development of reporting tools such as the Prostate Imaging Reporting and Data System (PI-RADS), diagnosing csPCa remains challenging, the authors explained. There is considerable interreader variability, and high expertise is required.
Various studies have shown that AI can potentially improve diagnostic performance, reduce interreader variability, and boost accurate diagnoses, yet most of these studies have been limited by small datasets and reader cohorts, they added.
Thus, to offer more robust evidence of AI’s potential, the researchers conducted a large international observer study. They hypothesized that the assistance of a high-performing AI system would lead to significantly improved csPCa diagnosis compared with reader assessments without AI support.
The AI system was developed in the previously conducted Prostate Imaging-Cancer AI (PI-CAI) Challenge, in which the system significantly helped improve diagnostic accuracy in interpreting prostate MRI exams. In this study, between March and July 2024, 61 readers evaluated 360 MRI exams culled from the PI-CAI Challenge both with and without AI assistance.
The readers provided Prostate Imaging Reporting and Data System (PI-RADS) annotations from 3 to 5 (higher PI-RADS indicates a higher likelihood of csPCa) and patient-level suspicion scores ranging from 0 to 100, with higher scores indicating a greater likelihood of harboring csPCa.
The readers had a median of five years of experience in reading prostate MRI and were familiar with PI-RADS, version 2.1, of whom 70% practiced in clinical routine and 30% were in residency. Using self-reporting, 34 readers (56%) were categorized as experts (> 1,000 cases read in total and > 200 cases per year), while the remaining 27 readers (44%) were categorized as nonexperts.
According to the analysis, AI assistance was associated with significantly improved performance by readers, who achieved a 3.3% increase in the area under the receiver operating characteristic curve from 0.882 in unassisted assessments to 0.916 with AI assistance.
In addition, reader sensitivity improved by 2.5% with AI assistance, from 94.3% to 96.8%, and in cases of PI-RADS scores of 3 or more, specificity increased by 3.4%, from 46.7% to 50.1%, the researchers reported.
Finally, sensitivity improvements were 3.7% for nonexperts versus 1.5% for experts, while specificity gains were 4.3% for nonexperts versus 2.8% for experts.
“Consistent with prior research, our findings support the role of AI assistance associated with improved csPCa diagnosis,” the researchers wrote.
The group noted that readers in the study assessed examinations through a controlled online reading workstation, which may have differed considerably from their native environments and may have impacted diagnostic performance. Moreover, the study did not assess workflow efficiency or the clinical applicability of improvements, which requires deployment in real or simulated clinical settings, they added.
“There is a need for continued exploration of human-AI interactions, along with the prospective deployment of AI in a clinical setting to assess the generalizability of our findings and to evaluate its impact on workflow efficiency,” the group concluded.
The full study is available here.