Large language models for generating medical examinations: systematic review.

BACKGROUND: Writing multiple choice questions (MCQs) for the purpose of medical exams is challenging. It requires extensive medical knowledge, time and effort from medical educators. This systematic review focuses on the application of large language models (LLMs) in generating medical MCQs. METHODS: The authors searched for studies published up to November 2023. Search terms focused on LLMs generated MCQs for medical examinations. Non-English, out of year range and studies not focusing on AI generated multiple-choice questions were excluded. MEDLINE was used as a search database. Risk of bias was evaluated using a tailored QUADAS-2 tool. RESULTS: Overall, eight studies published between April 2023 and October 2023 were included. Six studies used Chat-GPT 3.5, while two employed GPT 4. Five studies showed that LLMs can produce competent questions valid for medical exams. Three studies used LLMs to write medical questions but did not evaluate the validity of the questions. One study conducted a comparative analysis of different models. One other study compared LLM-generated questions with those written by humans. All studies presented faulty questions that were deemed inappropriate for medical exams. Some questions required additional modifications in order to qualify. CONCLUSIONS: LLMs can be used to write MCQs for medical examinations. However, their limitations cannot be ignored. Further study in this field is essential and more conclusive evidence is needed. Until then, LLMs may serve as a supplementary tool for writing medical examinations. 2 studies were at high risk of bias. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

Advancing Medical Practice with Artificial Intelligence: ChatGPT in Healthcare.

BACKGROUND: Advancements in artificial intelligence (AI) and natural language processing (NLP) have led to the development of language models such as ChatGPT. These models have the potential to transform healthcare and medical research. However, understanding their applications and limitations is essential. OBJECTIVES: To present a view of ChatGPT research and to critically assess ChatGPT's role in medical writing and clinical environments. METHODS: We performed a literature review via the PubMed search engine from 20 November 2022, to 23 April 2023. The search terms included ChatGPT, OpenAI, and large language models. We included studies that focused on ChatGPT, explored its use or implications in medicine, and were original research articles. The selected studies were analyzed considering study design, NLP tasks, main findings, and limitations. RESULTS: Our study included 27 articles that examined ChatGPT's performance in various tasks and medical fields. These studies covered knowledge assessment, writing, and analysis tasks. While ChatGPT was found to be useful in tasks such as generating research ideas, aiding clinical reasoning, and streamlining workflows, limitations were also identified. These limitations included inaccuracies, inconsistencies, fictitious information, and limited knowledge, highlighting the need for further improvements. CONCLUSIONS: The review underscores ChatGPT's potential in various medical applications. Yet, it also points to limitations that require careful human oversight and responsible use to improve patient care, education, and decision-making.

Alerting on mortality among patients discharged from the emergency department: a machine learning model.

OBJECTIVES: Physicians continuously make tough decisions when discharging patients. Alerting on poor outcomes may help in this decision. This study evaluates a machine learning model for predicting 30-day mortality in emergency department (ED) discharged patients. METHODS: We retrospectively analysed visits of adult patients discharged from a single ED (1/2014-12/2018). Data included demographics, evaluation and treatment in the ED, and discharge diagnosis. The data comprised of both structured and free-text fields. A gradient boosting model was trained to predict mortality within 30 days of release from the ED. The model was trained on data from the years 2014-2017 and validated on data from the year 2018. In order to reduce potential end-of-life bias, a subgroup analysis was performed for non-oncological patients. RESULTS: Overall, 363 635 ED visits of discharged patients were analysed. The 30-day mortality rate was 0.8%. A majority of the mortality cases (65.3%) had a known oncological disease. The model yielded an area under the curve (AUC) of 0.97 (95% CI 0.96 to 0.97) for predicting 30-day mortality. For a sensitivity of 84% (95% CI 0.81 to 0.86), this model had a false positive rate of 1:20. For patients without a known malignancy, the model yielded an AUC of 0.94 (95% CI 0.92 to 0.95). CONCLUSIONS: Although not frequent, patients may die following ED discharge. Machine learning-based tools may help ED physicians identify patients at risk. An optimised decision for hospitalisation or palliative management may improve patient care and system resource allocation.

[CONTRAST ENHANCED SPECTRAL MAMMOGRAPHY IN WOMEN WITH INTERMEDIATE BREAST CANCER RISK AND DENSE BREAST].

INTRODUCTION: Breast cancer screening decreases mortality and enables early diagnosis of breast cancer. Mammography is the only modality approved for breast cancer screening. Yet, mammography is limited in women with dense breasts. Contrast-enhanced mammography is a new imaging modality. OBJECTIVES: The aim of this study was to evaluate the diagnostic performance of contrast-enhanced mammography for breast cancer screening in women with dense breasts and intermediate breast cancer risk. The study strives to compare the diagnostic performance of contrast-enhanced mammography to standard digital mammography in women with intermediate-risk and dense breasts. METHODS: A retrospective cohort of 270 consecutive women who underwent screening with contrast mammography between the years 2015-2016. BI-RADS scores of both conventional and contrast-enhanced mammography were compared with the actual disease status, assessed by histopathology or imaging follow-up. Sensitivities, specificities, positive and negative predictive values were calculated. RESULTS: Conventional mammography detected 7 out of 11 breast cancers, with sensitivity of 63.6%, specificity 91.1%, positive predictive value 23.3% and negative predictive value of 98.3%. Contrast-enhanced mammography detected 10 out of 11 cancers. Sensitivity was 90.9%, specificity 70.7%, positive predictive value 11.6%, and negative predictive value 99.4. CONCLUSIONS: Contrast-enhanced mammography was more sensitive than digital mammography at detecting breast cancer in women with dense breasts and intermediate breast cancer risk. DISCUSSION: The technological development in breast imaging can be part of personalized medicine including contrast mammography for women with intermediate risk. Contrast mammography can be the screening examination for women with dense breasts and intermediate risk.

Refractive surgery in the late adulthood and adolescent age groups.

PURPOSE: Most refractive surgeries are performed in the young-adult age group, and less is known about the clinical outcomes of patients in late adulthood and of adolescents. The purpose of this study was to describe the outcomes of refractive surgery in patients over the age of 60 years and under the age of 18 years compared with a control group of patients aged 20-40 years. METHODS: This retrospective cohort analysis consisted of 64,970 consecutive cases of 32,074 patients who underwent laser-assisted in situ keratomileusis and photorefractive keratectomy during a 10-year period in a single center. The populations were characterized, and a comparison of safety, efficacy, and retreatment rates was performed following propensity score matching, separately for hyperopic and myopic treatments. RESULTS: Included in the analysis after matching were 143 patients above the age of 60, 608 patients aged < 18, and 2313 patients aged 20-40. Older patients undergoing hyperopic treatments had worse safety (0.95 ± 0.1 versus 0.99 ± 0.2, P = 0.023) and efficacy indices (0.89 ± 0.2 versus 0.97 ± 0.2, P = 0.004) compared with young adults. Lower efficacy was also seen in myopic treatments (0.88 ± 0.3 versus 0.97 ± 0.2, P = 0.001). Higher retreatment rates were also seen among older adults (6.2% versus 2.5%, P = 0.044 in hyperopic treatments, 11% versus 1.1%, P < 0.001 in myopic treatments). In adolescents, the safety and efficacy outcomes were slightly better compared with patients aged 20-40, with lower retreatment rates (1% versus 2.7%, P = 0.001). CONCLUSIONS: Refractive surgery in the late adulthood population of our cohort was a relatively safe procedure, yet manifesting lower efficacy and requiring more retreatments. In adolescents, results were comparable to those achieved in young adults.

Contrast-Enhanced Spectral Mammography in Women With Intermediate Breast Cancer Risk and Dense Breasts.

OBJECTIVE: The purpose of this study was to compare the diagnostic performance of contrast-enhanced spectral mammography (CESM) and ultrasound with that of standard digital mammography for breast cancer screening of women at intermediate risk who have dense breasts. MATERIALS AND METHODS: In a retrospective cohort of 611 consecutively registered women who underwent screening CESM from 2012 to 2017, BI-RADS scores of the screening modalities were compared with actual disease status, assessed by histopathologic analysis or imaging follow-up. Sensitivity, specificity, and positive and negative predictive values were calculated. RESULTS: Among the 611 women included, 48.3% (295/611) had family or personal history of breast cancer, the BI-RADS breast density score was C or D in 93.1% (569/611). The mean follow-up period was 20 months. Mammography depicted 11 of 21 malignancies, sensitivity of 52.4%, specificity of 90.5% (534/590), positive predictive value of 16.4% (11/67), and negative predictive value of 98.2% (534/544). CESM depicted 19 of 21 malignancies, sensitivity of 90.5%, specificity of 76.1% (449/590), positive predictive value of 11.9% (19/160), and negative predictive value of 99.6% (449/451). Differences in sensitivity (p = 0.008) and specificity (p < 0.001) were statistically significant. Adjunct ultrasound revealed 73 additional suspicious findings; all were false-positive. In 39 women MRI was needed to assess screening abnormalities; two MRI-guided biopsies were performed and yielded one cancer. The incremental cancer detection rate of CESM was 13.1/1000 women (95% CI, 6.1-20.1). Of eight cancers seen only with CESM, seven were invasive (mean size, 9 mm; two of four cancers lymph-node positive). CONCLUSION: CESM was significantly more sensitive than standard digital mammography for detecting breast cancer in this screening population. No added benefit was found in the performance of ultrasound as an adjunct to CESM screens with negative results. CESM may be a valuable supplemental screening modality for women at intermediate risk who have dense breasts.

Utilizing large language models in breast cancer management: systematic review.

PURPOSE: Despite advanced technologies in breast cancer management, challenges remain in efficiently interpreting vast clinical data for patient-specific insights. We reviewed the literature on how large language models (LLMs) such as ChatGPT might offer solutions in this field. METHODS: We searched MEDLINE for relevant studies published before December 22, 2023. Keywords included: "large language models", "LLM", "GPT", "ChatGPT", "OpenAI", and "breast". The risk bias was evaluated using the QUADAS-2 tool. RESULTS: Six studies evaluating either ChatGPT-3.5 or GPT-4, met our inclusion criteria. They explored clinical notes analysis, guideline-based question-answering, and patient management recommendations. Accuracy varied between studies, ranging from 50 to 98%. Higher accuracy was seen in structured tasks like information retrieval. Half of the studies used real patient data, adding practical clinical value. Challenges included inconsistent accuracy, dependency on the way questions are posed (prompt-dependency), and in some cases, missing critical clinical information. CONCLUSION: LLMs hold potential in breast cancer care, especially in textual information extraction and guideline-driven clinical question-answering. Yet, their inconsistent accuracy underscores the need for careful validation of these models, and the importance of ongoing supervision.

Artificial Intelligence for Identification of Images with Active Bleeding in Mesenteric and Celiac Arteries Angiography.

PURPOSE: The purpose of this study is to evaluate the efficacy of an artificial intelligence (AI) model designed to identify active bleeding in digital subtraction angiography images for upper gastrointestinal bleeding. METHODS: Angiographic images were retrospectively collected from mesenteric and celiac artery embolization procedures performed between 2018 and 2022. This dataset included images showing both active bleeding and non-bleeding phases from the same patients. The images were labeled as normal versus images that contain active bleeding. A convolutional neural network was trained and validated to automatically classify the images. Algorithm performance was tested in terms of area under the curve, accuracy, sensitivity, specificity, F1 score, positive and negative predictive value. RESULTS: The dataset included 587 pre-labeled images from 142 patients. Of these, 302 were labeled as normal angiogram and 285 as containing active bleeding. The model's performance on the validation cohort was area under the curve 85.0 ± 10.9% (standard deviation) and average classification accuracy 77.43 ± 4.9%. For Youden's index cutoff, sensitivity and specificity were 85.4 ± 9.4% and 81.2 ± 8.6%, respectively. CONCLUSION: In this study, we explored the application of AI in mesenteric and celiac artery angiography for detecting active bleeding. The results of this study show the potential of an AI-based algorithm to accurately classify images with active bleeding. Further studies using a larger dataset are needed to improve accuracy and allow segmentation of the bleeding.

Artificial intelligence for detection of effusion and lipo-hemarthrosis in X-rays and CT of the knee.

BACKGROUND: Traumatic knee injuries are challenging to diagnose accurately through radiography and to a lesser extent, through CT, with fractures sometimes overlooked. Ancillary signs like joint effusion or lipo-hemarthrosis are indicative of fractures, suggesting the need for further imaging. Artificial Intelligence (AI) can automate image analysis, improving diagnostic accuracy and help prioritizing clinically important X-ray or CT studies. OBJECTIVE: To develop and evaluate an AI algorithm for detecting effusion of any kind in knee X-rays and selected CT images and distinguishing between simple effusion and lipo-hemarthrosis indicative of intra-articular fractures. METHODS: This retrospective study analyzed post traumatic knee imaging from January 2016 to February 2023, categorizing images into lipo-hemarthrosis, simple effusion, or normal. It utilized the FishNet-150 algorithm for image classification, with class activation maps highlighting decision-influential regions. The AI's diagnostic accuracy was validated against a gold standard, based on the evaluations made by a radiologist with at least four years of experience. RESULTS: Analysis included CT images from 515 patients and X-rays from 637 post traumatic patients, identifying lipo-hemarthrosis, simple effusion, and normal findings. The AI showed an AUC of 0.81 for detecting any effusion, 0.78 for simple effusion, and 0.83 for lipo-hemarthrosis in X-rays; and 0.89, 0.89, and 0.91, respectively, in CTs. CONCLUSION: The AI algorithm effectively detects knee effusion and differentiates between simple effusion and lipo-hemarthrosis in post-traumatic patients for both X-rays and selected CT images further studies are needed to validate these results.

Navigating the vestibular maze: text-mining analysis of publication trends over five decades.

Introduction: The field of vestibular science, encompassing the study of the vestibular system and associated disorders, has experienced notable growth and evolving trends over the past five decades. Here, we explore the changing landscape in vestibular science, focusing on epidemiology, peripheral pathologies, diagnosis methods, treatment, and technological advancements. Methods: Publication data was obtained from the US National Center for Biotechnology Information (NCBI) PubMed database. The analysis included epidemiological, etiological, diagnostic, and treatment-focused studies on peripheral vestibular disorders, with a particular emphasis on changes in topics and trends of publications over time. Results: Our dataset of 39,238 publications revealed a rising trend in research across all age groups. Etiologically, benign paroxysmal positional vertigo (BPPV) and Meniere's disease were the most researched conditions, but the prevalence of studies on vestibular migraine showed a marked increase in recent years. Electronystagmography (ENG)/ Videonystagmography (VNG) and Vestibular Evoked Myogenic Potential (VEMP) were the most commonly discussed diagnostic tools, while physiotherapy stood out as the primary treatment modality. Conclusion: Our study presents a unique opportunity and point of view, exploring the evolving landscape of vestibular science publications over the past five decades. The analysis underscored the dynamic nature of the field, highlighting shifts in focus and emerging publication trends in diagnosis and treatment over time.

Breast cancer screening in BRCA1/2 pathogenic sequence variant carriers during pregnancy and lactation.

OBJECTIVES: Women harboring germline BRCA1/BRCA2 pathogenic sequence variants (PSVs) are at an increased risk for breast cancer. There are no established guidelines for screening during pregnancy and lactation in BRCA carriers. The aim of this study was to evaluate the utility of whole-breast ultrasound (US) screening in pregnant and lactating BRCA PSV carriers. METHODS: Data were retrospectively collected from medical records of BRCA PSV carriers between 2014 and 2020, with follow-up until 2021. Associations between imaging intervals, number of examinations performed and pregnancy-associated breast cancers (PABCs) were examined. PABCs and cancers diagnosed at follow-up were evaluated and characteristics were compared between the two groups. RESULTS: Overall 212 BRCA PSV carriers were included. Mean age was 33.6 years (SD 3.93, range 25-43 years). During 274 screening periods at pregnancy and lactation, eight (2.9 %) PABCs were diagnosed. An additional eight cancers were diagnosed at follow-up. Three out of eight (37.5 %) PABCs were diagnosed by US, whereas clinical breast examination (n = 3), mammography (n = 1) and MRI (n = 1) accounted for the other PACB diagnoses. One PABC was missed by US. The interval from negative imaging to cancer diagnosis was significantly shorter for PABCs compared with cancers diagnosed at follow-up (3.96 ± 2.14 vs. 11.2 ± 4.46 months, P = 0.002). CONCLUSION: In conclusion, pregnant BRCA PSV carriers should not delay screening despite challenges like altered breast tissue and hesitancy towards mammography. If no alternatives exist, whole-breast ultrasound can be used. For lactating and postpartum women, a regular screening routine alternating between mammography and MRI is recommended.

Large language models and the emergence phenomena.

This perspective explores the potential of emergence phenomena in large language models (LLMs) to transform data management and analysis in radiology. We provide a concise explanation of LLMs, define the concept of emergence in machine learning, offer examples of potential applications within the radiology field, and discuss risks and limitations. Our goal is to encourage radiologists to recognize and prepare for the impact this technology may have on radiology and medicine in the near future.

ChatGPT-4 Assistance in Optimizing Emergency Department Radiology Referrals and Imaging Selection.

PURPOSE: The quality of radiology referrals influences patient management and imaging interpretation by radiologists. The aim of this study was to evaluate ChatGPT-4 as a decision support tool for selecting imaging examinations and generating radiology referrals in the emergency department (ED). METHODS: Five consecutive clinical notes from the ED were retrospectively extracted, for each of the following pathologies: pulmonary embolism, obstructing kidney stones, acute appendicitis, diverticulitis, small bowel obstruction, acute cholecystitis, acute hip fracture, and testicular torsion. A total of 40 cases were included. These notes were entered into ChatGPT-4, requesting recommendations on the most appropriate imaging examinations and protocols. The chatbot was also asked to generate radiology referrals. Two independent radiologists graded the referral on a scale ranging from 1 to 5 for clarity, clinical relevance, and differential diagnosis. The chatbot's imaging recommendations were compared with the ACR Appropriateness Criteria (AC) and with the examinations performed in the ED. Agreement between readers was assessed using linear weighted Cohen's κ coefficient. RESULTS: ChatGPT-4's imaging recommendations aligned with the ACR AC and ED examinations in all cases. Protocol discrepancies between ChatGPT and the ACR AC were observed in two cases (5%). ChatGPT-4-generated referrals received mean scores of 4.6 and 4.8 for clarity, 4.5 and 4.4 for clinical relevance, and 4.9 from both reviewers for differential diagnosis. Agreement between readers was moderate for clinical relevance and clarity and substantial for differential diagnosis grading. CONCLUSIONS: ChatGPT-4 has shown potential in aiding imaging study selection for select clinical cases. As a complementary tool, large language models may improve radiology referral quality. Radiologists should stay informed about this technology and be mindful of potential challenges and risks.

Large language models for oncological applications.

Large language models such as ChatGPT have gained public and scientific attention. These models may support oncologists in their work. Oncologists should be familiar with large language models to harness their potential while being aware of potential dangers and limitations.

Adversarial attacks in radiology - A systematic review.

PURPOSE: The growing application of deep learning in radiology has raised concerns about cybersecurity, particularly in relation to adversarial attacks. This study aims to systematically review the literature on adversarial attacks in radiology. METHODS: We searched for studies on adversarial attacks in radiology published up to April 2023, using MEDLINE and Google Scholar databases. RESULTS: A total of 22 studies published between March 2018 and April 2023 were included, primarily focused on image classification algorithms. Fourteen studies evaluated white-box attacks, three assessed black-box attacks and five investigated both. Eleven of the 22 studies targeted chest X-ray classification algorithms, while others involved chest CT (6/22), brain MRI (4/22), mammography (2/22), abdominal CT (1/22), hepatic US (1/22), and thyroid US (1/22). Some attacks proved highly effective, reducing the AUC of algorithm performance to 0 and achieving success rates up to 100 %. CONCLUSIONS: Adversarial attacks are a growing concern. Although currently the threats are more theoretical than practical, they still represent a potential risk. It is important to be alert to such attacks, reinforce cybersecurity measures, and influence the formulation of ethical and legal guidelines. This will ensure the safe use of deep learning technology in medicine.

Treatment-Induced Bone Marrow Enhancement at MRI in Breast Cancer Patients Receiving Granulocyte-Colony Stimulating Factor Adjunct to Chemotherapy.

RATIONALE AND OBJECTIVES: Granulocyte-colony stimulating factor (G-CSF) induces the reconversion of fatty bone marrow to hematopoietic bone marrow. The bone marrow changes are detectable as signal intensity changes at MRI. The aim of this study was to evaluate sternal bone marrow enhancement following G-CSF and chemotherapy treatment in women with breast cancer. MATERIALS AND METHODS: This retrospective study included breast cancer patients who received neoadjuvant chemotherapy with adjunct G-CSF. The signal intensity of sternal bone marrow at MRI on T1-weighted contrast-enhanced subtracted images was measured before treatment, at the end of treatment, and at 1-year follow-up. The bone marrow signal intensity (BM SI) index was calculated by dividing the signal intensity of sternal marrow by the signal intensity of the chest wall muscle. Data were collected between 2012 and 2017, with follow-up until August 2022. Mean BM SI indices were compared before and after treatment, and at 1-year follow-up. Differences in bone marrow enhancement between time points were analyzed using a one-way repeated measures ANOVA. RESULTS: A total of 109 breast cancer patients (mean age 46.1 ± 10.4 years) were included in our study. None of the women had distal metastases at presentation. A repeated-measures ANOVA determined that mean BM SI index scores differed significantly across the three time points (F[1.62, 100.67] = 44.57, p < .001). At post hoc pairwise comparison using the Bonferroni correction BM SI index significantly increased between initial assessment and following treatment (2.15 vs 3.33, p < .001), and significantly decreased at 1-year follow-up (3.33 vs 1.45, p < .001). In a subgroup analysis, while women younger than 50 years had a significant increase in marrow enhancement after G-CSF treatment, in women aged 50 years and older, the difference was not statistically significant. CONCLUSION: Treatment with G-CSF as an adjunct to chemotherapy can result in increased sternal bone marrow enhancement due to marrow reconversion. Radiologists should be aware of this effect in order to avoid misinterpretation as false marrow metastases.