Evaluating the performance of Generative Pre-trained Transformer-4 (GPT-4) in standardizing radiology reports.

OBJECTIVE: Radiology reporting is an essential component of clinical diagnosis and decision-making. With the advent of advanced artificial intelligence (AI) models like GPT-4 (Generative Pre-trained Transformer 4), there is growing interest in evaluating their potential for optimizing or generating radiology reports. This study aimed to compare the quality and content of radiologist-generated and GPT-4 AI-generated radiology reports. METHODS: A comparative study design was employed in the study, where a total of 100 anonymized radiology reports were randomly selected and analyzed. Each report was processed by GPT-4, resulting in the generation of a corresponding AI-generated report. Quantitative and qualitative analysis techniques were utilized to assess similarities and differences between the two sets of reports. RESULTS: The AI-generated reports showed comparable quality to radiologist-generated reports in most categories. Significant differences were observed in clarity (p = 0.027), ease of understanding (p = 0.023), and structure (p = 0.050), favoring the AI-generated reports. AI-generated reports were more concise, with 34.53 fewer words and 174.22 fewer characters on average, but had greater variability in sentence length. Content similarity was high, with an average Cosine Similarity of 0.85, Sequence Matcher Similarity of 0.52, BLEU Score of 0.5008, and BERTScore F1 of 0.8775. CONCLUSION: The results of this proof-of-concept study suggest that GPT-4 can be a reliable tool for generating standardized radiology reports, offering potential benefits such as improved efficiency, better communication, and simplified data extraction and analysis. However, limitations and ethical implications must be addressed to ensure the safe and effective implementation of this technology in clinical practice. CLINICAL RELEVANCE STATEMENT: The findings of this study suggest that GPT-4 (Generative Pre-trained Transformer 4), an advanced AI model, has the potential to significantly contribute to the standardization and optimization of radiology reporting, offering improved efficiency and communication in clinical practice. KEY POINTS: • Large language model-generated radiology reports exhibited high content similarity and moderate structural resemblance to radiologist-generated reports. • Performance metrics highlighted the strong matching of word selection and order, as well as high semantic similarity between AI and radiologist-generated reports. • Large language model demonstrated potential for generating standardized radiology reports, improving efficiency and communication in clinical settings.

Ultra-Small Lung Cysts Impair Diffusion Without Obstructing Air Flow in Lymphangioleiomyomatosis.

BACKGROUND: Lymphangioleiomyomatosis (LAM) is a rare lung disease found primarily in women of childbearing age, characterized by the formation of air-filled cysts, which may be associated with reductions in lung function. An experimental, regional ultra-high resolution CT scan identified an additional volume of cysts relative to standard chest CT imaging, which consisted primarily of ultra-small cysts. RESEARCH QUESTION: What is the impact of these ultra-small cysts on the pulmonary function of patients with LAM? STUDY DESIGN AND METHODS: A group of 103 patients with LAM received pulmonary function tests and a CT examination in the same visit. Cyst score, the percentage lung volume occupied by cysts, was measured by using commercial software approved by the US Food and Drug Administration. The association between cyst scores and pulmonary function tests of diffusing capacity of the lungs for carbon monoxide (Dlco) (% predicted), FEV1 (% predicted), and FEV1/FVC (% predicted) was assessed with statistical analysis adjusted for demographic variables. The distributions of average cyst size and ultra-small cyst fraction among the patients were evaluated. RESULTS: The additional cyst volume identified by the experimental, higher resolution scan consisted of cysts of 2.2 ± 0.8 mm diameter on average and are thus labeled the "ultra-small cyst fraction." It accounted for 27.9 ± 19.0% of the total cyst volume among the patients. The resulting adjusted, whole-lung cyst scores better explained the variance of Dlco (P < .001 adjusted for multiple comparisons) but not FEV1 and FEV1/FVC (P = 1.00). The ultra-small cyst fraction contributed to the reduction in Dlco (P < .001) but not to FEV1 and FEV1/FVC (P = .760 and .575, respectively). The ultra-small cyst fraction and average cyst size were correlated with cyst burden, FEV1, and FEV1/FVC but less with Dlco. INTERPRETATION: The ultra-small cysts primarily contributed to the reduction in Dlco, with minimal effects on FEV1 and FEV1/FVC. Patients with lower cyst burden and better FEV1 and FEV1/FVC tended to have smaller average cyst size and higher ultra-small cyst fraction. CLINICAL TRIAL REGISTRATION: ClinicalTrials.gov; No.: NCT00001465; URL: www.clinicaltrials.gov.

Clinical CT underestimation of the percentage volume occupied by cysts in patients with lymphangioleiomyomatosis.

PURPOSE: To evaluate the accuracy of cyst score measurements by standard high-resolution helical volume chest CT (HRCT) in patients with lymphangioleiomyomatosis (LAM), using a short z-length ultra-high resolution re-scan (UH re-scan) as the reference. In cystic lung diseases, cyst score is derived from CT scans and defined as the percentage of the total lung parenchymal volume occupied by cysts, a biomarker which measures the severity of the disease. METHODS: In a prospective study of 73 LAM patients, each patient received the standard HRCT chest scan and a short z-length UH re-scan. Cyst scores were acquired from both scans using a standard FDA-approved scoring software on the CT scanner. RESULTS: The limited UH re-scan resolved small cysts that were not resolved in the HRCT. The HRCT-derived cyst scores were on average 59.6% of the reference values from the UH re-scan (p = 4.7e-25). The amount of under-estimation by HRCT varied from patient to patient, with an inter-quartile range of 29.8% and standard deviation of 20.7%. The overall trend was more pronounced underestimation for patients with lower cyst scores. For patients whose reference cyst scores were below 15 (n = 29), the HRCT cyst scores were 46.9 ± 21.6% of reference values (p = 7.4e-12), while for the rest of the patients (n = 44) the HRCT cyst scores were 68.0 ± 15.3% of reference values (p = 1.2E-19). Reconstructing the HRCT images to the resolution of the UH re-scan further widened the spread of the discrepancy between HRCT and reference values due to increased image noise, and did not provide accurate cyst scores. CONCLUSION: Cyst scores derived from standard high-resolution helical volume chest CT significantly underestimates the percentage lung volume occupied by cysts. This inaccuracy needs to be taken into consideration when cyst score is used as part of the CT assessment of the patient's condition.