Quantitative Evaluation of Large Language Models to Streamline Radiology Report Impressions: A Multimodal Retrospective Analysis.

Background The complex medical terminology of radiology reports may cause confusion or anxiety for patients, especially given increased access to electronic health records. Large language models (LLMs) can potentially simplify radiology report readability. Purpose To compare the performance of four publicly available LLMs (ChatGPT-3.5 and ChatGPT-4, Bard [now known as Gemini], and Bing) in producing simplified radiology report impressions. Materials and Methods In this retrospective comparative analysis of the four LLMs (accessed July 23 to July 26, 2023), the Medical Information Mart for Intensive Care (MIMIC)-IV database was used to gather 750 anonymized radiology report impressions covering a range of imaging modalities (MRI, CT, US, radiography, mammography) and anatomic regions. Three distinct prompts were employed to assess the LLMs' ability to simplify report impressions. The first prompt (prompt 1) was "Simplify this radiology report." The second prompt (prompt 2) was "I am a patient. Simplify this radiology report." The last prompt (prompt 3) was "Simplify this radiology report at the 7th grade level." Each prompt was followed by the radiology report impression and was queried once. The primary outcome was simplification as assessed by readability score. Readability was assessed using the average of four established readability indexes. The nonparametric Wilcoxon signed-rank test was applied to compare reading grade levels across LLM output. Results All four LLMs simplified radiology report impressions across all prompts tested (P < .001). Within prompts, differences were found between LLMs. Providing the context of being a patient or requesting simplification at the seventh-grade level reduced the reading grade level of output for all models and prompts (except prompt 1 to prompt 2 for ChatGPT-4) (P < .001). Conclusion Although the success of each LLM varied depending on the specific prompt wording, all four models simplified radiology report impressions across all modalities and prompts tested. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Rahsepar in this issue.

Combating Chronic Disease with Barbershop Health Interventions: A Review of Current Knowledge and Potential for Big Data.

Community-based participatory research (CBPR) using barbershop interventions is an emerging approach to address health disparities and promote health equity. Barbershops serve as trusted community settings for health education, screening services, and referrals. This narrative mini-review provides an overview of the current state of knowledge regarding CBPR employing barbershop interventions and explores the potential for big data involvement to enhance the impact and reach of this approach in combating chronic disease. CBPR using barbershop interventions has shown promising results in reducing blood pressure among Black men and improving diabetes awareness and self-management. By increasing testing rates and promoting preventive behaviors, barbershop interventions have been successful in addressing infectious diseases, including HIV and COVID-19. Barbershops have also played roles in promoting cancer screening and increasing awareness of cancer risks, namely prostate cancer and colorectal cancer. Further, leveraging the trusted relationships between barbers and their clients, mental health promotion and prevention efforts have been successful in barbershops. The potential for big data involvement in barbershop interventions for chronic disease management offers new opportunities for targeted programs, real-time monitoring, and personalized approaches. However, ethical considerations regarding privacy, confidentiality, and data ownership need to be carefully addressed. To maximize the impact of barbershop interventions, challenges such as training and resource provision for barbers, cultural appropriateness of interventions, sustainability, and scalability must be addressed. Further research is needed to evaluate long-term impact, cost-effectiveness, and best practices for implementation. Overall, barbershops have the potential to serve as key partners in addressing chronic health disparities and promoting health equity.

Assessing the Efficacy of Large Language Models in Health Literacy: A Comprehensive Cross-Sectional Study.

Enhanced health literacy in children has been empirically linked to better health outcomes over the long term; however, few interventions have been shown to improve health literacy. In this context, we investigate whether large language models (LLMs) can serve as a medium to improve health literacy in children. We tested pediatric conditions using 26 different prompts in ChatGPT-3.5, ChatGPT-4, Microsoft Bing, and Google Bard (now known as Google Gemini). The primary outcome measurement was the reading grade level (RGL) of output as assessed by Gunning Fog, Flesch-Kincaid Grade Level, Automated Readability Index, and Coleman-Liau indices. Word counts were also assessed. Across all models, output for basic prompts such as "Explain" and "What is (are)," were at, or exceeded, the tenth-grade RGL. When prompts were specified to explain conditions from the first- to twelfth-grade level, we found that LLMs had varying abilities to tailor responses based on grade level. ChatGPT-3.5 provided responses that ranged from the seventh-grade to college freshmen RGL while ChatGPT-4 outputted responses from the tenth-grade to the college senior RGL. Microsoft Bing provided responses from the ninth- to eleventh-grade RGL while Google Bard provided responses from the seventh- to tenth-grade RGL. LLMs face challenges in crafting outputs below a sixth-grade RGL. However, their capability to modify outputs above this threshold, provides a potential mechanism for adolescents to explore, understand, and engage with information regarding their health conditions, spanning from simple to complex terms. Future studies are needed to verify the accuracy and efficacy of these tools.

Artificial Intelligence to Improve Patient Understanding of Radiology Reports.

Diagnostic imaging reports are generally written with a target audience of other providers. As a result, the reports are written with medical jargon and technical detail to ensure accurate communication. With implementation of the 21st Century Cures Act, patients have greater and quicker access to their imaging reports, but these reports are still written above the comprehension level of the average patient. Consequently, many patients have requested reports to be conveyed in language accessible to them. Numerous studies have shown that improving patient understanding of their condition results in better outcomes, so driving comprehension of imaging reports is essential. Summary statements, second reports, and the inclusion of the radiologist's phone number have been proposed, but these solutions have implications for radiologist workflow. Artificial intelligence (AI) has the potential to simplify imaging reports without significant disruptions. Many AI technologies have been applied to radiology reports in the past for various clinical and research purposes, but patient focused solutions have largely been ignored. New natural language processing technologies and large language models (LLMs) have the potential to improve patient understanding of their imaging reports. However, LLMs are a nascent technology and significant research is required before LLM-driven report simplification is used in patient care.

Even with ChatGPT, race matters.

BACKGROUND: Applications of large language models such as ChatGPT are increasingly being studied. Before these technologies become entrenched, it is crucial to analyze whether they perpetuate racial inequities. METHODS: We asked Open AI's ChatGPT-3.5 and ChatGPT-4 to simplify 750 radiology reports with the prompt "I am a ___ patient. Simplify this radiology report:" while providing the context of the five major racial classifications on the U.S. census: White, Black or African American, American Indian or Alaska Native, Asian, and Native Hawaiian or other Pacific Islander. To ensure an unbiased analysis, the readability scores of the outputs were calculated and compared. RESULTS: Statistically significant differences were found in both models based on the racial context. For ChatGPT-3.5, output for White and Asian was at a significantly higher reading grade level than both Black or African American and American Indian or Alaska Native, among other differences. For ChatGPT-4, output for Asian was at a significantly higher reading grade level than American Indian or Alaska Native and Native Hawaiian or other Pacific Islander, among other differences. CONCLUSION: Here, we tested an application where we would expect no differences in output based on racial classification. Hence, the differences found are alarming and demonstrate that the medical community must remain vigilant to ensure large language models do not provide biased or otherwise harmful outputs.

Does Baseline Mental Health Influence Outcomes among Workers' Compensation Claimants Undergoing Minimally Invasive Transforaminal Lumbar Interbody Fusion?

STUDY DESIGN: This was a retrospective cohort study. PURPOSE: This study investigated the influence of preoperative mental health on patient-reported outcome measures (PROMs) and minimal clinically important difference (MCID) among workers' compensation (WC) recipients undergoing minimally invasive transforaminal lumbar interbody fusion (MIS TLIF). OVERVIEW OF LITERATURE: No studies have evaluated the impact of preoperative mental functioning on outcomes following MIS TLIF among WC claimants. METHODS: WC recipients undergoing single-level MIS TLIF were identified. PROMs of Visual Analog Scale (VAS) for back and leg pain, Oswestry Disability Index (ODI), 12-item Short Form Physical and Mental Composite Scale (SF-12 PCS/MCS), and Patient-Reported Outcomes Measurement Information System Physical Function evaluated subjects preoperatively/postoperatively. Subjects were grouped according to preoperative SF-12 MCS: <41 vs. ≥41. Demographic/perioperative variables, PROMs, and MCID were compared using inferential statistics. Multiple regression was used to account for differences in spinal pathology. RESULTS: The SF-12 MCS <41 and SF-12 MCS ≥41 groups included 48 and 45 patients, respectively. Significant differences in ΔPROMs were observed at SF-12 MCS at all timepoints, except at 6 months (p≤0.041, all). The SF-12 MCS <41 group had worse preoperative to 6-months SF-12 MCS, 12-weeks/6-months VAS back, 12-week VAS leg, and preoperative to 6-months ODI (p≤0.029, all). The SF-12 MCS <41 group had greater MCID achievement for overall ODI and 6-weeks/1-year/overall SF-12 MCS (p≤0.043, all); the SF-12 MCS ≥41 group had greater attainment for 6-month VAS back (p=0.004). CONCLUSIONS: Poorer mental functioning adversely affected the baseline and intermediate postoperative quality-of-life outcomes pertaining to mental health, back pain, and disability among WC recipients undergoing lumbar fusion. However, outcomes did not differ 1-2 years after surgery. While MCID achievement for pain and physical function was largely unaffected by preoperative mental health score, WC recipients with poorer baseline mental health demonstrated higher rates of overall clinically meaningful improvements for disability and mental health.

Role of Psychologists in Pediatric Congenital Heart Disease.

Congenital heart disease (CHD) is stressful to both pediatric patients and their caregivers. Maternal anxiety during pregnancy is associated with adverse perinatal outcomes. After birth, a prolonged hospital stay can be taxing on the infant and caregiver leading to long-term adverse effects. During adolescence, CHD continues to serve as a stressor for the child not only due to medical care but also due to social limitations and bullying. Many patients also struggle during the transition from adolescence to adult care. Psychologists may aid both the parents and child at all stages from pregnancy to the child's transition to adulthood.

The Effect of the Preoperative Severity of Neck Pain on Patient-Reported Outcome Measures and Minimum Clinically Important Difference Achievement After Anterior Cervical Discectomy and Fusion.

OBJECTIVE: To compare patient-reported outcome measure (PROM) scores and minimum clinically important difference (MCID) achievement rates among patients undergoing single-level anterior cervical discectomy and fusion (ACDF) in patients with varying severity of preoperative visual analog scale (VAS) neck score. METHODS: Patients with ACDF were grouped: severity of preoperative VAS neck score ≤8 or >8. Demographic/perioperative variables and PROMs (Patient-Reported Outcomes Measurement Information System Physical Function [PROMIS PF] score, 12-Item Short Form [SF-12] Mental Component Score [MCS], VAS neck/arm score, and Neck Disability Index [NDI]) were collected preoperatively/postoperatively. MCID attainment comparison by grouping was evaluated using χ2 analysis. RESULTS: A total of 137 patients were included (103 VAS neck preoperative score ≤8; 34 VAS neck preoperative score >8). The VAS neck preoperative score ≤8 cohort did not improve: 6 weeks PROMIS-PF score, 6 weeks SF-12 Physical Component Score [PCS], 12 weeks/1 year/2 years SF-12 MCS, 2 years VAS neck score, and 1 years/2 years VAS arm score (P ≤ 0.015, all). VAS neck preoperative score >8 did not improve: 6 weeks/12 weeks/2 years PROMIS-PF score, all time points SF-12 PCS, 6 weeks/12 weeks/1 year/2 years SF-12 MCS, and 2 years VAS arm score (P ≤ 0.013, all). VAS neck preoperative score >8 had inferior PROMIS-PF scores all time points except 1 year (P ≤ 0.036, all), lower SF-12 PCS 6 weeks/6 months (P ≤ 0.043, both), inferior SF-12 MCS at preoperative to 6 months (P ≤ 006, all), higher VAS neck score from preoperative to 6 months (P ≤ 0.018), higher VAS arm score preoperative/12 weeks/6 months (P ≤ 0.020, all), and higher NDI at preoperative/12 weeks/6 months (P ≤ 0.030, all). MCID attainment rates for VAS neck preoperative score >8 were greater for NDI 2 years (P = 0.040), lower for PROMIS-PF score 2 years, and overall (P = 0.018), lower for SF-12 MCS 12 weeks (P = 0.046), lower for VAS neck score 12 weeks to 1 year and overall (P ≤ 0.032, all), and lower for VAS arm score 6 weeks/1 year (P ≤ 0.030, both). CONCLUSIONS: Patients with single-level ACDF presenting with greater baseline neck pain showed poorer physical function/pain/disability/mental health at preoperative/intermediate postoperative time points, but had comparable long-term PROMs by 2 years. MCID attainment was lower among patients with greater preoperative neck pain; MCID among the VAS neck score >8 cohort were only significantly inferior for neck pain.

Obesity and Workers' Compensation in the Setting of Minimally Invasive Lumbar Decompression.

OBJECTIVE: To investigate the influence of body mass index (BMI) on perioperative outcomes, postoperative patient-reported outcome measures (PROMs), and minimal clinically important difference (MCID) achievement among workers' compensation (WC) claimants undergoing minimally invasive lumbar decompression (MIS-LD). METHODS: WC patients diagnosed with herniated nucleus pulposus undergoing single-level MIS-LD were identified. Patients were divided into 3 groups: Non-obese (<30 kg/m2), Obese I (≥30 and <35 kg/m2), and Obese II/III (≥35 kg/m2). PROMs were collected preoperatively and at 6 weeks, 12 weeks, 6 months, 1 year, and 2 years postoperatively. The predictive influence of BMI grouping on mean PROM scores was computed using simple linear regression. To compare PROMs between groups, post hoc pairwise comparisons of adjusted means were utilized. MCID achievement was compared between groups with χ2 analysis. RESULTS: A total of 81 patients were in the Non-obese cohort, and 43 and 45 in the Obese I and Obese II/III cohorts, respectively. Visual analog scale (VAS) leg, Oswestry Disability Index (ODI), and 12-Item Short Form Physical Composite Score (SF-12 PCS) were worse in the Obese I cohort at 12 weeks, and SF-12 PCS was lower in the Obese I vs. Obese II/III subgroup analysis (P ≤ 0.045, all). MCID achievement rates for ODI were higher for the Non-obese group at 12 weeks and overall (P ≤ 0.049, both). MCID attainment for VAS back was higher among the Non-obese cohort at 6-weeks (P = 0.022). CONCLUSIONS: Patients with higher levels of obesity were more likely to experience longer length of stay and delayed discharge following MIS-LD. Increasing BMI was generally not a significant predictor of postoperative pain, disability, or physical health PROMs at most timepoints. MCID achievement rates for disability relief were significantly higher for non-obese patients.

Reticulocyte hemoglobin content as a function of iron stores at 35-36 weeks post menstrual age in very premature infants.

BACKGROUND: Premature infants are born with lower iron stores and are at risk for iron deficiency during early infancy. To prevent iron deficiency, premature infants are routinely supplemented with 2 mg/kg/day oral elemental iron. Reticulocyte hemoglobin content (RET-He), a measure of iron deficiency, has not been well evaluated prior to discharge in premature infants. OBJECTIVES: Our objectives were to evaluate RET-He and its correlation with serum ferritin (SF), an index of iron stores, at 35-36 weeks postmenstrual age (PMA) in ≤32 weeks gestational age (GA) infants. METHODS: We performed a prospective nested study involving 24-32 weeks GA infants who were receiving 2 mg/kg/day oral elemental iron with full enteral feedings at 35-36 weeks PMA. Infants with the following conditions were excluded: craniofacial malformation, chromosomal disorders, TORCH (toxoplasmosis, other infections, rubella, cytomegalovirus, and herpes simplex), culture-proven sepsis, C-reactive protein >5 mg/l within 10 days of iron status assessment, or erythropoietin therapy. SF and RET-He were measured at 35-36 weeks PMA using chemiluminescence immunoassay and Sysmex XN hematology analyzer, respectively. RET-He <27 pg was deemed indicative of iron deficiency. RESULTS: Ninety-eight infants were studied, of which 21 infants had RET-He <27 pg. There was a positive correlation between RET-He and SF (coefficient 0.22, p = .03) that remained significant after controlling for GA (coefficient 0.21, p = .03) and frequency of prior erythrocyte transfusions (coefficient 0.21, p = .03). On stratified analysis, there was a positive correlation between SF and RET-He in females (N = 52, coefficient 0.23, p = .02), but not in males (N = 46, coefficient 0.05). CONCLUSIONS: Most premature infants receiving 2 mg/kg/day oral elemental iron are iron replete for erythropoiesis at 35-36 weeks PMA. RET-He increases with an increase in iron stores, suggesting that additional iron supplementation prior to discharge to very premature infants with borderline low RET-He may help prevent iron deficiency during early infancy.