US of thyroid nodules: can AI-assisted diagnostic system compete with fine needle aspiration?

ABSTRACT

OBJECTIVES: Artificial intelligence (AI) systems can diagnose thyroid nodules with similar or better performance than radiologists. Little is known about how this performance compares with that achieved through fine needle aspiration (FNA). This study aims to compare the diagnostic yields of FNA cytopathology alone and combined with BRAFV600E mutation analysis and an AI diagnostic system. METHODS: The ultrasound images of 637 thyroid nodules were collected in three hospitals. The diagnostic efficacies of an AI diagnostic system, FNA-based cytopathology, and BRAFV600E mutation analysis were evaluated in terms of sensitivity, specificity, accuracy, and the κ coefficient with respect to the gold standard, defined by postsurgical pathology and consistent benign outcomes from two combined FNA and mutation analysis examinations performed with a half-year interval. RESULTS: The malignancy threshold for the AI system was selected according to the Youden index from a retrospective cohort of 346 nodules and then applied to a prospective cohort of 291 nodules. The combination of FNA cytopathology according to the Bethesda criteria and BRAFV600E mutation analysis showed no significant difference from the AI system in terms of accuracy for either cohort in our multicenter study. In addition, for 45 included indeterminate Bethesda category III and IV nodules, the accuracy, sensitivity, and specificity of the AI system were 84.44%, 95.45%, and 73.91%, respectively. CONCLUSIONS: The AI diagnostic system showed similar diagnostic performance to FNA cytopathology combined with BRAFV600E mutation analysis. Given its advantages in terms of operability, time efficiency, non-invasiveness, and the wide availability of ultrasonography, it provides a new alternative for thyroid nodule diagnosis. CLINICAL RELEVANCE STATEMENT Thyroid ultrasonic artificial intelligence shows statistically equivalent performance for thyroid nodule diagnosis to FNA cytopathology combined with BRAFV600E mutation analysis. It can be widely applied in hospitals and clinics to assist radiologists in thyroid nodule screening and is expected to reduce the need for relatively invasive FNA biopsies. KEY POINTS • In a retrospective cohort of 346 nodules, the evaluated artificial intelligence (AI) system did not significantly differ from fine needle aspiration (FNA) cytopathology alone and combined with gene mutation analysis in accuracy. • In a prospective multicenter cohort of 291 nodules, the accuracy of the AI diagnostic system was not significantly different from that of FNA cytopathology either alone or combined with gene mutation analysis. • For 45 indeterminate Bethesda category III and IV nodules, the AI system did not perform significantly differently from BRAFV600E mutation analysis.