Modified American College of Radiology Thyroid Imaging Reporting and Data System and Modified Artificial Intelligence Thyroid Imaging Reporting and Data System for Thyroid Nodules: A Multicenter Retrospective Study.

Background: Risk stratification systems for thyroid nodules are limited by low specificity. The fine-needle aspiration (FNA) biopsy size thresholds and stratification criteria are based on evidence from the literature and expert consensus. Our aims were to investigate the optimal FNA biopsy size thresholds in the American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS) and artificial intelligence (AI) TI-RADS and to revise the stratification criteria in AI TI-RADS. Methods: A total of 2596 thyroid nodules (in 2511 patients) on ultrasound examination with definite pathological diagnoses were retrospectively identified from January 2017 to September 2021 in 6 participating Chinese hospitals. The modified criteria for ACR TI-RADS were as follows: (1) no FNA for TR3; (2) FNA threshold for TR4 increased to 2.5 cm. The modified criteria for AI TI-RADS were as follows: (1) 6-point nodules upgraded to TR5; (2) no FNA for TR3; (3) FNA threshold for TR4 increased to 2.5 cm. The diagnostic performance and the unnecessary FNA rate (UFR) of modified versions were compared with the original ACR TI-RADS. Results: Compared with the original ACR TI-RADS, the modified ACR (mACR) TI-RADS yielded higher specificity (73% vs. 46%), accuracy (74% vs. 51%), area under the receiver operating characteristic curve (AUC; 0.80 vs. 0.70), and lower UFR (25% vs. 48%; all p < 0.001), although the sensitivity was slightly decreased (87% vs. 93%, p = 0.057). Compared with the original ACR TI-RADS, the modified AI (mAI) TI-RADS yielded higher specificity (73% vs. 46%), accuracy (75% vs. 51%), AUC (0.81 vs. 0.70), and lower UFR (24% vs. 48%; all p < 0.001), although the sensitivity tended to be slightly decreased (89% vs. 93%, p = 0.13). There was no significant difference between the mACR TI-RADS and mAI TI-RADS in the diagnostic performance and UFR (all p > 0.05). Conclusions: The revised FNA thresholds and the stratification criteria of the mACR TI-RADS and mAI TI-RADS may be associated with improvements in specificity and accuracy, without significantly sacrificing sensitivity for malignancy detection.

Comparison of diagnostic accuracy and utility of artificial intelligence-optimized ACR TI-RADS and original ACR TI-RADS: a multi-center validation study based on 2061 thyroid nodules.

OBJECTIVE: To determine if artificial intelligence-based modification of the Thyroid Imaging Reporting Data System (TI-RADS) would be better than the current American College of Radiology (ACR) TI-RADS for risk stratification of thyroid nodules. METHODS: A total of 2061 thyroid nodules (in 1859 patients) sampled with fine-needle aspiration or operation were retrospectively analyzed between January 2017 and July 2020. Two radiologists blinded to the pathologic diagnosis evaluated nodule features in five ultrasound categories and assigned TI-RADS scores by both ACR TI-RADS and AI TI-RADS. Inter-rater agreement was assessed by asking another two radiologists to score a set of 100 nodules independently. The reference standard was postoperative pathological or cytopathological diagnosis according to the Bethesda system. Inter-rater agreement was determined using intraclass correlation coefficient (ICC). RESULTS: AI TI-RADS assigned lower TI-RADS risk levels than ACR TI-RADS (p < 0.001) and had larger area under receiver operating characteristic curve (0.762 vs. 0.679, p < 0.001). The sensitivities of ACR TI-RADS and AI TI-RADS were similar (86.7% vs. 82.2%, p = 0.052), but specificity was higher with AI TI-RADS (70.2% vs. 49.2%, p < 0.001). AI TI-RADS downgraded 743 (48.63%) benign nodules, indicating that 328 (42.3% of 776 biopsied nodules) unnecessary fine-needle aspirations (FNA) could have been avoided. Inter-rater agreement was better with AI TI-RADS than with ACR TI-RADS (ICC, 0.808 vs. 0.861, p < 0.001). CONCLUSION: AI TI-RADS can achieve meaningful reduction in the number of benign thyroid nodules recommended for biopsy and significantly improve specificity despite a slight decrease in sensitivity. KEY POINTS: • AI TI-RADS assigned lower TI-RADS risk levels than ACR TI-RADS, showing similar sensitivity but higher specificity. • Half of the benign nodules can be downgraded of which 42.3% of biopsy nodules avoided unnecessary fine-needle aspiration (FNA). • AI TI-RADS had a better overall inter-rater agreement.