Methodological insights into ChatGPT's screening performance in systematic reviews.

BACKGROUND: The screening process for systematic reviews and meta-analyses in medical research is a labor-intensive and time-consuming task. While machine learning and deep learning have been applied to facilitate this process, these methods often require training data and user annotation. This study aims to assess the efficacy of ChatGPT, a large language model based on the Generative Pretrained Transformers (GPT) architecture, in automating the screening process for systematic reviews in radiology without the need for training data. METHODS: A prospective simulation study was conducted between May 2nd and 24th, 2023, comparing ChatGPT's performance in screening abstracts against that of general physicians (GPs). A total of 1198 abstracts across three subfields of radiology were evaluated. Metrics such as sensitivity, specificity, positive and negative predictive values (PPV and NPV), workload saving, and others were employed. Statistical analyses included the Kappa coefficient for inter-rater agreement, ROC curve plotting, AUC calculation, and bootstrapping for p-values and confidence intervals. RESULTS: ChatGPT completed the screening process within an hour, while GPs took an average of 7-10 days. The AI model achieved a sensitivity of 95% and an NPV of 99%, slightly outperforming the GPs' sensitive consensus (i.e., including records if at least one person includes them). It also exhibited remarkably low false negative counts and high workload savings, ranging from 40 to 83%. However, ChatGPT had lower specificity and PPV compared to human raters. The average Kappa agreement between ChatGPT and other raters was 0.27. CONCLUSIONS: ChatGPT shows promise in automating the article screening phase of systematic reviews, achieving high sensitivity and workload savings. While not entirely replacing human expertise, it could serve as an efficient first-line screening tool, particularly in reducing the burden on human resources. Further studies are needed to fine-tune its capabilities and validate its utility across different medical subfields.

Differential privacy preserved federated learning for prognostic modeling in COVID-19 patients using large multi-institutional chest CT dataset.

BACKGROUND: Notwithstanding the encouraging results of previous studies reporting on the efficiency of deep learning (DL) in COVID-19 prognostication, clinical adoption of the developed methodology still needs to be improved. To overcome this limitation, we set out to predict the prognosis of a large multi-institutional cohort of patients with COVID-19 using a DL-based model. PURPOSE: This study aimed to evaluate the performance of deep privacy-preserving federated learning (DPFL) in predicting COVID-19 outcomes using chest CT images. METHODS: After applying inclusion and exclusion criteria, 3055 patients from 19 centers, including 1599 alive and 1456 deceased, were enrolled in this study. Data from all centers were split (randomly with stratification respective to each center and class) into a training/validation set (70%/10%) and a hold-out test set (20%). For the DL model, feature extraction was performed on 2D slices, and averaging was performed at the final layer to construct a 3D model for each scan. The DensNet model was used for feature extraction. The model was developed using centralized and FL approaches. For FL, we employed DPFL approaches. Membership inference attack was also evaluated in the FL strategy. For model evaluation, different metrics were reported in the hold-out test sets. In addition, models trained in two scenarios, centralized and FL, were compared using the DeLong test for statistical differences. RESULTS: The centralized model achieved an accuracy of 0.76, while the DPFL model had an accuracy of 0.75. Both the centralized and DPFL models achieved a specificity of 0.77. The centralized model achieved a sensitivity of 0.74, while the DPFL model had a sensitivity of 0.73. A mean AUC of 0.82 and 0.81 with 95% confidence intervals of (95% CI: 0.79-0.85) and (95% CI: 0.77-0.84) were achieved by the centralized model and the DPFL model, respectively. The DeLong test did not prove statistically significant differences between the two models (p-value = 0.98). The AUC values for the inference attacks fluctuate between 0.49 and 0.51, with an average of 0.50 ± 0.003 and 95% CI for the mean AUC of 0.500 to 0.501. CONCLUSION: The performance of the proposed model was comparable to centralized models while operating on large and heterogeneous multi-institutional datasets. In addition, the model was resistant to inference attacks, ensuring the privacy of shared data during the training process.

Accuracy of artificial intelligence CT quantification in predicting COVID-19 subjects' prognosis.

BACKGROUND: Artificial intelligence (AI)-aided analysis of chest CT expedites the quantification of abnormalities and may facilitate the diagnosis and assessment of the prognosis of subjects with COVID-19. OBJECTIVES: This study investigates the performance of an AI-aided quantification model in predicting the clinical outcomes of hospitalized subjects with COVID-19 and compares it with radiologists' performance. SUBJECTS AND METHODS: A total of 90 subjects with COVID-19 (men, n = 59 [65.6%]; age, 52.9±16.7 years) were recruited in this cross-sectional study. Quantification of the total and compromised lung parenchyma was performed by two expert radiologists using a volumetric image analysis software and compared against an AI-assisted package consisting of a modified U-Net model for segmenting COVID-19 lesions and an off-the-shelf U-Net model augmented with COVID-19 data for segmenting lung volume. The fraction of compromised lung parenchyma (%CL) was calculated. Based on clinical results, the subjects were divided into two categories: critical (n = 45) and noncritical (n = 45). All admission data were compared between the two groups. RESULTS: There was an excellent agreement between the radiologist-obtained and AI-assisted measurements (intraclass correlation coefficient = 0.88, P < 0.001). Both the AI-assisted and radiologist-obtained %CLs were significantly higher in the critical subjects (P = 0.009 and 0.02, respectively) than in the noncritical subjects. In the multivariate logistic regression analysis to distinguish the critical subjects, an AI-assisted %CL ≥35% (odds ratio [OR] = 17.0), oxygen saturation level of <88% (OR = 33.6), immunocompromised condition (OR = 8.1), and other comorbidities (OR = 15.2) independently remained as significant variables in the models. Our proposed model obtained an accuracy of 83.9%, a sensitivity of 79.1%, and a specificity of 88.6% in predicting critical outcomes. CONCLUSIONS: AI-assisted measurements are similar to quantitative radiologist-obtained measurements in determining lung involvement in COVID-19 subjects.

Evaluation of chest CT-scan appearances of COVID-19 according to RSNA classification system.

Background: The Radiologic Society of North America (RSNA) divides patients into four sections: negative, atypical, indeterminate, and typical coronavirus disease 2019 (COVID-19) pneumonia based on their computed tomography (CT) scan findings. Herein, we evaluate the frequency of the chest CT-scan appearances of COVID-19 according to each RSNA categorical group. Methods: A total of 90 patients with real-time reverse transcriptase-polymerase chain reaction (RT-PCR)-confirmed COVID-19 were enrolled in this study and differences in age, sex, cardiac characteristics, and imaging features of lung parenchyma were evaluated in different categories of RSNA classification. Results: According to the RSNA classification 87.8, 5.56, 4.44, and 2.22% of the patients were assigned as typical, indeterminate, atypical, and negative, respectively. The proportion of "atypical" patients was higher in the patients who had mediastinal lymphadenopathy and pleural effusion. Moreover, ground-glass opacity (GGO) and consolidation were more pronounced in the lower lobes and left lung compared to the upper lobes and right lung, respectively. While small nodules were mostly seen in the atypical group, small GGO was associated with the typical group, especially when it is present in the right lung and indeterminate group. Conclusion: Regardless of its location, non-round GGO is the most prevalent finding in the typical group of the RSNA classification systems. Mediastinal lymphadenopathy, pleural effusion, and small nodules are mostly observed in the atypical group and small GGO in the right lung is mostly seen in the typical group.

COVID-19 prognostic modeling using CT radiomic features and machine learning algorithms: Analysis of a multi-institutional dataset of 14,339 patients.

BACKGROUND: We aimed to analyze the prognostic power of CT-based radiomics models using data of 14,339 COVID-19 patients. METHODS: Whole lung segmentations were performed automatically using a deep learning-based model to extract 107 intensity and texture radiomics features. We used four feature selection algorithms and seven classifiers. We evaluated the models using ten different splitting and cross-validation strategies, including non-harmonized and ComBat-harmonized datasets. The sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were reported. RESULTS: In the test dataset (4,301) consisting of CT and/or RT-PCR positive cases, AUC, sensitivity, and specificity of 0.83 ± 0.01 (CI95%: 0.81-0.85), 0.81, and 0.72, respectively, were obtained by ANOVA feature selector + Random Forest (RF) classifier. Similar results were achieved in RT-PCR-only positive test sets (3,644). In ComBat harmonized dataset, Relief feature selector + RF classifier resulted in the highest performance of AUC, reaching 0.83 ± 0.01 (CI95%: 0.81-0.85), with a sensitivity and specificity of 0.77 and 0.74, respectively. ComBat harmonization did not depict statistically significant improvement compared to a non-harmonized dataset. In leave-one-center-out, the combination of ANOVA feature selector and RF classifier resulted in the highest performance. CONCLUSION: Lung CT radiomics features can be used for robust prognostic modeling of COVID-19. The predictive power of the proposed CT radiomics model is more reliable when using a large multicentric heterogeneous dataset, and may be used prospectively in clinical setting to manage COVID-19 patients.

Chest CT Scan Features to Predict COVID-19 Patients' Outcome and Survival.

Background: Providing efficient care for infectious coronavirus disease 2019 (COVID-19) patients requires an accurate and accessible tool to medically optimize medical resource allocation to high-risk patients. Purpose: To assess the predictive value of on-admission chest CT characteristics to estimate COVID-19 patients' outcome and survival time. Materials and Methods: Using a case-control design, we included all laboratory-confirmed COVID-19 patients who were deceased, from June to September 2020, in a tertiary-referral-collegiate hospital and had on-admission chest CT as the case group. The patients who did not die and were equivalent in terms of demographics and other clinical features to cases were considered as the control (survivors) group. The equivalency evaluation was performed by a fellowship-trained radiologist and an expert radiologist. Pulmonary involvement (PI) was scored (0-25) using a semiquantitative scoring tool. The PI density index was calculated by dividing the total PI score by the number of involved lung lobes. All imaging parameters were compared between case and control group members. Survival time was recorded for the case group. All demographic, clinical, and imaging variables were included in the survival analyses. Results: After evaluating 384 cases, a total of 186 patients (93 in each group) were admitted to the studied setting, consisting of 126 (67.7%) male patients with a mean age of 60.4 ± 13.6 years. The PI score and PI density index in the case vs. the control group were on average 8.9 ± 4.5 vs. 10.7 ± 4.4 (p value: 0.001) and 2.0 ± 0.7 vs. 2.6 ± 0.8 (p value: 0.01), respectively. Axial distribution (p value: 0.01), cardiomegaly (p value: 0.005), pleural effusion (p value: 0.001), and pericardial effusion (p value: 0.04) were mostly observed in deceased patients. Our survival analyses demonstrated that PI score ≥ 10 (p value: 0.02) and PI density index ≥ 2.2 (p value: 0.03) were significantly associated with a lower survival rate. Conclusion: On-admission chest CT features, particularly PI score and PI density index, are potential great tools to predict the patient's clinical outcome.

Clinical and chest CT features as a predictive tool for COVID-19 clinical progress: introducing a novel semi-quantitative scoring system.

OBJECTIVE: Proposing a scoring tool to predict COVID-19 patients' outcomes based on initially assessed clinical and CT features. METHODS: All patients, who were referred to a tertiary-university hospital respiratory triage (March 27-April 26, 2020), were highly clinically suggestive for COVID-19 and had undergone a chest CT scan were included. Those with positive rRT-PCR or highly clinically suspicious patients with typical chest CT scan pulmonary manifestations were considered confirmed COVID-19 for additional analyses. Patients, based on outcome, were categorized into outpatient, ordinary-ward admitted, intensive care unit (ICU) admitted, and deceased; their demographic, clinical, and chest CT scan parameters were compared. The pulmonary chest CT scan features were scaled with a novel semi-quantitative scoring system to assess pulmonary involvement (PI). RESULTS: Chest CT scans of 739 patients (mean age = 49.2 ± 17.2 years old, 56.7% male) were reviewed; 491 (66.4%), 176 (23.8%), and 72 (9.7%) cases were managed outpatient, in an ordinary ward, and ICU, respectively. A total of 439 (59.6%) patients were confirmed COVID-19 cases; their most prevalent chest CT scan features were ground-glass opacity (GGO) (93.3%), pleural-based peripheral distribution (60.3%), and multi-lobar (79.7%), bilateral (76.6%), and lower lobes (RLL and/or LLL) (89.1%) involvement. Patients with lower SpO2, advanced age, RR, total PI score or PI density score, and diffuse distribution or involvement of multi-lobar, bilateral, or lower lobes were more likely to be ICU admitted/expired. After adjusting for confounders, predictive models found cutoffs of age ≥ 53, SpO2 ≤ 91, and PI score ≥ 8 (15) for ICU admission (death). A combination of all three factors showed 89.1% and 95% specificity and 81.9% and 91.4% accuracy for ICU admission and death outcomes, respectively. Solely evaluated high PI score had high sensitivity, specificity, and NPV in predicting the outcome as well. CONCLUSION: We strongly recommend patients with age ≥ 53, SpO2 ≤ 91, and PI score ≥ 8 or even only high PI score to be considered as high-risk patients for further managements and care plans. KEY POINTS: • Chest CT scan is a valuable tool in prioritizing the patients in hospital triage. • A more accurate and novel 35-scale semi-quantitative scoring system was designed to predict the COVID-19 patients' outcome. • Patients with age ≥ 53, SpO2 ≤ 91, and PI score ≥ 8 or even only high PI score should be considered high-risk patients.