RESUMO
OBJECTIVES: The artificial intelligence competition in healthcare at TEKNOFEST-2022 provided a platform to address the complex multi-class classification challenge of abdominal emergencies using computer vision techniques. This manuscript aimed to comprehensively present the methodologies for data preparation, annotation procedures, and rigorous evaluation metrics. Moreover, it was conducted to introduce a meticulously curated abdominal emergencies data set to the researchers. METHODS: The data set underwent a comprehensive central screening procedure employing diverse algorithms extracted from the e-Nabiz (Pulse) and National Teleradiology System of the Republic of Türkiye, Ministry of Health. Full anonymization of the data set was conducted. Subsequently, the data set was annotated by a group of ten experienced radiologists. The evaluation process was executed by calculating F1 scores, which were derived from the intersection over union values between the predicted bounding boxes and the corresponding ground truth (GT) bounding boxes. The establishment of baseline performance metrics involved computing the average of the highest five F1 scores. RESULTS: Observations indicated a progressive decline in F1 scores as the threshold value increased. Furthermore, it could be deduced that class 6 (abdominal aortic aneurysm/dissection) was relatively straightforward to detect compared to other classes, with class 5 (acute diverticulitis) presenting the most formidable challenge. It is noteworthy, however, that if all achieved outcomes for all classes were considered with a threshold of 0.5, the data set's complexity and associated challenges became pronounced. CONCLUSION: This data set's significance lies in its pioneering provision of labels and GT-boxes for six classes, fostering opportunities for researchers. CLINICAL RELEVANCE STATEMENT: The prompt identification and timely intervention in cases of emergent medical conditions hold paramount significance. The handling of patients' care can be augmented, while the potential for errors is minimized, particularly amidst high caseload scenarios, through the application of AI. KEY POINTS: ⢠The data set used in artificial intelligence competition in healthcare (TEKNOFEST-2022) provides a 6-class data set of abdominal CT images consisting of a great variety of abdominal emergencies. ⢠This data set is compiled from the National Teleradiology System data repository of emergency radiology departments of 459 hospitals. ⢠Radiological data on abdominal emergencies is scarce in literature and this annotated competition data set can be a valuable resource for further studies and new AI models.
RESUMO
OBJECTIVE: The artificial intelligence competition in healthcare was organized for the first time at the annual aviation, space, and technology festival (TEKNOFEST), Istanbul/Türkiye, in September 2021. In this article, the data set preparation and competition processes were explained in detail; the anonymized and annotated data set is also provided via official website for further research. MATERIALS AND METHODS: Data set recorded over the period covering 2019 and 2020 were centrally screened from the e-Pulse and Teleradiology System of the Republic of Türkiye, Ministry of Health using various codes and filtering criteria. The data set was anonymized. The data set was prepared, pooled, curated, and annotated by 7 radiologists. The training data set was shared with the teams via a dedicated file transfer protocol server, which could be accessed using private usernames and passwords given to the teams under a nondisclosure agreement signed by the representative of each team. RESULTS: The competition consisted of 2 stages. In the first stage, teams were given 192 digital imaging and communications in medicine images that belong to 1 of 3 possible categories namely, hemorrhage, ischemic, or non-stroke. Teams were asked to classify each image as either stroke present or absent. In the second stage of the competition, qualifying 36 teams were given 97 digital imaging and communications in medicine images that contained hemorrhage, ischemia, or both lesions. Among the employed methods, Unet and DeepLabv3 were the most frequently observed ones. CONCLUSION: Artificial intelligence competitions in healthcare offer good opportunities to collect data reflecting various cases and problems. Especially, annotated data set by domain experts is more valuable.
