Artificial intelligence and radiologists in prostate cancer detection on MRI (PI-CAI): an international, paired, non-inferiority, confirmatory study

Author(s): Anindo Saha, MSc1,2; Joeran S Bosma, MSc1; Jasper J Twilt, MSc2; Prof Bram van Ginneken, PhD1; Prof Anders Bjartell, MD3,4; Prof Anwar R Padhani, MD5; Prof David Bonekamp, MD6; Prof Geert Villeirs, MD7; Prof Georg Salomon, MD8; Prof Gianluca Giannarini, MD9; Prof Jayashree Kalpathy-Cramer, PhD10; Prof Jelle Barentsz, MD11; Prof Klaus H Maier-Hein, PhD12,13; Mirabela Rusu, PhD14; Prof Olivier Rouvière, MD15,16; Roderick van den Bergh, MD17; Prof Valeria Panebianco, MD18; Veeru Kasivisvanathan, MD19; Prof Nancy A Obuchowski, PhD20; Derya Yakar, MD21,22; Mattijs Elschot, PhD23,24; Jeroen Veltman, MD25,26; Prof Jurgen J Fütterer, MD2; Maarten de Rooij, MD27; Prof Henkjan Huisman, PhD1,23
Source: DOI:https://doi.org/10.1016/S1470-2045(24)00220-1

Dr. Anjan Patel's Thoughts

Will we be replaced by AI? It seems that for radiologists, an AI system was better than the human counterpart in this study looking at prostate cancer diagnostic imaging. I doubt many of us would accept a solely computer-generated report, but this study highlights how AI may help as a supportive tool in the primary diagnostic setting. Of course, prospective validation will be needed.

BACKGROUND

Artificial intelligence (AI) systems can potentially aid the diagnostic pathway of prostate cancer by alleviating the increasing workload, preventing overdiagnosis, and reducing the dependence on experienced radiologists. We aimed to investigate the performance of AI systems at detecting clinically significant prostate cancer on MRI in comparison with radiologists using the Prostate Imaging—Reporting and Data System version 2.1 (PI-RADS 2.1) and the standard of care in multidisciplinary routine practice at scale.

METHODS

In this international, paired, non-inferiority, confirmatory study, we trained and externally validated an AI system (developed within an international consortium) for detecting Gleason grade group 2 or greater cancers using a retrospective cohort of 10 207 MRI examinations from 9129 patients. Of these examinations, 9207 cases from three centres (11 sites) based in the Netherlands were used for training and tuning, and 1000 cases from four centres (12 sites) based in the Netherlands and Norway were used for testing. In parallel, we facilitated a multireader, multicase observer study with 62 radiologists (45 centres in 20 countries; median 7 [IQR 5–10] years of experience in reading prostate MRI) using PI-RADS (2.1) on 400 paired MRI examinations from the testing cohort. Primary endpoints were the sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC) of the AI system in comparison with that of all readers using PI-RADS (2.1) and in comparison with that of the historical radiology readings made during multidisciplinary routine practice (ie, the standard of care with the aid of patient history and peer consultation). Histopathology and at least 3 years (median 5 [IQR 4–6] years) of follow-up were used to establish the reference standard. The statistical analysis plan was prespecified with a primary hypothesis of non-inferiority (considering a margin of 0·05) and a secondary hypothesis of superiority towards the AI system, if non-inferiority was confirmed. This study was registered at ClinicalTrials.gov, NCT05489341.

FINDINGS

Of the 10 207 examinations included from Jan 1, 2012, through Dec 31, 2021, 2440 cases had histologically confirmed Gleason grade group 2 or greater prostate cancer. In the subset of 400 testing cases in which the AI system was compared with the radiologists participating in the reader study, the AI system showed a statistically superior and non-inferior AUROC of 0·91 (95% CI 0·87–0·94; p<0·0001), in comparison to the pool of 62 radiologists with an AUROC of 0·86 (0·83–0·89), with a lower boundary of the two-sided 95% Wald CI for the difference in AUROC of 0·02. At the mean PI-RADS 3 or greater operating point of all readers, the AI system detected 6·8% more cases with Gleason grade group 2 or greater cancers at the same specificity (57·7%, 95% CI 51·6–63·3), or 50·4% fewer false-positive results and 20·0% fewer cases with Gleason grade group 1 cancers at the same sensitivity (89·4%, 95% CI 85·3–92·9). In all 1000 testing cases where the AI system was compared with the radiology readings made during multidisciplinary practice, non-inferiority was not confirmed, as the AI system showed lower specificity (68·9% [95% CI 65·3–72·4] vs 69·0% [65·5–72·5]) at the same sensitivity (96·1%, 94·0–98·2) as the PI-RADS 3 or greater operating point. The lower boundary of the two-sided 95% Wald CI for the difference in specificity (−0·04) was greater than the non-inferiority margin (−0·05) and a p value below the significance threshold was reached (p<0·001).

INTERPRETATION

An AI system was superior to radiologists using PI-RADS (2.1), on average, at detecting clinically significant prostate cancer and comparable to the standard of care. Such a system shows the potential to be a supportive tool within a primary diagnostic setting, with several associated benefits for patients and radiologists. Prospective validation is needed to test clinical applicability of this system.

FUNDING

Health~Holland and EU Horizon 2020.

Author Affiliations

1Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, Netherlands; 2Minimally Invasive Image-Guided Intervention Center, Radboud University Medical Center, Nijmegen, Netherlands; 3Department of Urology, Skåne University Hospital, Malmö, Sweden; 4Division of Translational Cancer Research, Lund University Cancer Centre, Lund, Sweden; 5Paul Strickland Scanner Centre, Mount Vernon Cancer Centre, London, UK; 6Division of Radiology, Deutsches Krebsforschungszentrum Heidelberg, Heidelberg, Germany; 7Department of Diagnostic Sciences, Ghent University Hospital, Ghent, Belgium; 8Martini Clinic, Prostate Cancer Center, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany; 9Urology Unit, Santa Maria della Misericordia University Hospital, Udine, Italy; 10Division of Artificial Medical Intelligence in Ophthalmology, University of Colorado, Aurora, CO, USA; 11Department of Medical Imaging, Andros Clinics, Arnhem, Netherlands; 12Division of Medical Image Computing, Deutsches Krebsforschungszentrum Heidelberg, Heidelberg, Germany; 13Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany; 14Departments of Radiology, Urology and Biomedical Data Science, Stanford University, Stanford, CA, USA; 15Department of Urinary and Vascular Imaging, Hôpital Edouard Herriot, Hospices Civils de Lyon, Lyon, France; 16Faculté de Médecine Lyon-Est, Université de Lyon, Lyon, France; 17Department of Urology, Erasmus Medical Center, Rotterdam, Netherlands; 18Department of Radiological Sciences, Oncology and Pathology, Sapienza University of Rome, Rome, Italy; 19Division of Surgery and Interventional Sciences, University College London and University College London Hospital, London, UK; 20Department of Quantitative Health Sciences and Department of Diagnostic Radiology, Cleveland Clinic Foundation, Cleveland OH, USA; 21Department of Radiology, University Medical Center Groningen, Netherlands; 22Department of Radiology, Netherlands Cancer Institute, Amsterdam, Netherlands; 23Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Tronheim, Norway; 24Department of Radiology and Nuclear Medicine, St Olavs Hospital, Trondheim University Hospital, Trondheim, Norway; 25Department of Radiology, Ziekenhuisgroep Twente, Hengelo, Netherlands; 26Department of Multi-Modality Medical Imaging, Technical Medical Centre, University of Twente, Enschede, Netherlands; 27Department of Medical Imaging, Radboud University Medical Center, Nijmegen, Netherland

Leave a Comment

Your email address will not be published. Required fields are marked *

Related Articles

BRCAAway: A randomized phase 2 trial of abiraterone, olaparib, or abiraterone + olaparib in patients with metastatic castration-resistant prostate cancer (mCRPC) bearing homologous recombination-repair mutations (HRRm).

We know that combination PARP and antiandrogen is better than antiandrogen alone. I have wondered if we need antiancrogen on those patients. This shows that, YES, we do. The combination also was better than PARP inhibitor monotherapy in those patients carrying HRRm.

Read More »