Extracting structured information from unstructured histopathology reports using generative pre-trained transformer 4 (GPT-4).

Deep learning applied to whole-slide histopathology images (WSIs) has the potential to enhance precision oncology and alleviate the workload of experts. However, developing these models necessitates large amounts of data with ground truth labels, which can be both time-consuming and expensive to obtain. Pathology reports are typically unstructured or poorly structured texts, and efforts to implement structured reporting templates have been unsuccessful, as these efforts lead to perceived extra workload. In this study, we hypothesised that large language models (LLMs), such as the generative pre-trained transformer 4 (GPT-4), can extract structured data from unstructured plain language reports using a zero-shot approach without requiring any re-training. We tested this hypothesis by utilising GPT-4 to extract information from histopathological reports, focusing on two extensive sets of pathology reports for colorectal cancer and glioblastoma. We found a high concordance between LLM-generated structured data and human-generated structured data. Consequently, LLMs could potentially be employed routinely to extract ground truth data for machine learning from unstructured pathology reports in the future. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Artificial intelligence and machine learning in axial spondyloarthritis.

PURPOSE OF REVIEW: To evaluate the current applications and prospects of artificial intelligence and machine learning in diagnosing and managing axial spondyloarthritis (axSpA), focusing on their role in medical imaging, predictive modelling, and patient monitoring. RECENT FINDINGS: Artificial intelligence, particularly deep learning, is showing promise in diagnosing axSpA assisting with X-ray, computed tomography (CT) and MRI analyses, with some models matching or outperforming radiologists in detecting sacroiliitis and markers. Moreover, it is increasingly being used in predictive modelling of disease progression and personalized treatment, and could aid risk assessment, treatment response and clinical subtype identification. Variable study designs, sample sizes and the predominance of retrospective, single-centre studies still limit the generalizability of results. SUMMARY: Artificial intelligence technologies have significant potential to advance the diagnosis and treatment of axSpA, providing more accurate, efficient and personalized healthcare solutions. However, their integration into clinical practice requires rigorous validation, ethical and legal considerations, and comprehensive training for healthcare professionals. Future advances in artificial intelligence could complement clinical expertise and improve patient care through improved diagnostic accuracy and tailored therapeutic strategies, but the challenge remains to ensure that these technologies are validated in prospective multicentre trials and ethically integrated into patient care.

International pharmacy students' perceptions towards artificial intelligence in medicine-A multinational, multicentre cross-sectional study.

AIMS: To explore international undergraduate pharmacy students' views on integrating artificial intelligence (AI) into pharmacy education and practice. METHODS: This cross-sectional institutional review board-approved multinational, multicentre study comprised an anonymous online survey of 14 multiple-choice items to assess pharmacy students' preferences for AI events in the pharmacy curriculum, the current state of AI education, and students' AI knowledge and attitudes towards using AI in the pharmacy profession, supplemented by 8 demographic queries. Subgroup analyses were performed considering sex, study year, tech-savviness, and prior AI knowledge and AI events in the curriculum using the Mann-Whitney U-test. Variances were reported for responses in Likert scale format. RESULTS: The survey gathered 387 pharmacy student opinions across 16 faculties and 12 countries. Students showed predominantly positive attitudes towards AI in medicine (58%, n = 225) and expressed a strong desire for more AI education (72%, n = 276). However, they reported limited general knowledge of AI (63%, n = 242) and felt inadequately prepared to use AI in their future careers (51%, n = 197). Male students showed more positive attitudes towards increasing efficiency through AI (P = .011), while tech-savvy and advanced-year students expressed heightened concerns about potential legal and ethical issues related to AI (P < .001/P = .025, respectively). Students who had AI courses as part of their studies reported better AI knowledge (P < .001) and felt more prepared to apply it professionally (P < .001). CONCLUSIONS: Our findings underline the generally positive attitude of international pharmacy students towards AI application in medicine and highlight the necessity for a greater emphasis on AI education within pharmacy curricula.

Correction: Integrating Text and Image Analysis: Exploring GPT-4V's Capabilities in Advanced Radiological Applications Across Subspecialties.

[This corrects the article DOI: 10.2196/54948.].

Integrating Text and Image Analysis: Exploring GPT-4V's Capabilities in Advanced Radiological Applications Across Subspecialties.

This study demonstrates that GPT-4V outperforms GPT-4 across radiology subspecialties in analyzing 207 cases with 1312 images from the Radiological Society of North America Case Collection.

Multimodal Deep Learning for Integrating Chest Radiographs and Clinical Parameters: A Case for Transformers.

Background Clinicians consider both imaging and nonimaging data when diagnosing diseases; however, current machine learning approaches primarily consider data from a single modality. Purpose To develop a neural network architecture capable of integrating multimodal patient data and compare its performance to models incorporating a single modality for diagnosing up to 25 pathologic conditions. Materials and Methods In this retrospective study, imaging and nonimaging patient data were extracted from the Medical Information Mart for Intensive Care (MIMIC) database and an internal database comprised of chest radiographs and clinical parameters inpatients in the intensive care unit (ICU) (January 2008 to December 2020). The MIMIC and internal data sets were each split into training (n = 33 893, n = 28 809), validation (n = 740, n = 7203), and test (n = 1909, n = 9004) sets. A novel transformer-based neural network architecture was trained to diagnose up to 25 conditions using nonimaging data alone, imaging data alone, or multimodal data. Diagnostic performance was assessed using area under the receiver operating characteristic curve (AUC) analysis. Results The MIMIC and internal data sets included 36 542 patients (mean age, 63 years ± 17 [SD]; 20 567 male patients) and 45 016 patients (mean age, 66 years ± 16; 27 577 male patients), respectively. The multimodal model showed improved diagnostic performance for all pathologic conditions. For the MIMIC data set, the mean AUC was 0.77 (95% CI: 0.77, 0.78) when both chest radiographs and clinical parameters were used, compared with 0.70 (95% CI: 0.69, 0.71; P < .001) for only chest radiographs and 0.72 (95% CI: 0.72, 0.73; P < .001) for only clinical parameters. These findings were confirmed on the internal data set. Conclusion A model trained on imaging and nonimaging data outperformed models trained on only one type of data for diagnosing multiple diseases in patients in an ICU setting. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Kitamura and Topol in this issue.

Sex Differences in Renal Cell Carcinoma: The Importance of Body Composition.

PURPOSE: To examine sex-specific differences in renal cell carcinoma (RCC) in relation to abdominal fat accumulation, psoas muscle density, tumor size, pathology, and survival, and to evaluate possible associations with RCC characteristics and outcome. METHODS: A total of 470 patients with RCC who underwent nephrectomy between 2006 and 2019 were included in this retrospective study. Specific characteristics of RCC patients were collected, including sex, height, tumor size, grade, and data on patient survival, if available. Abdominal fat measurements and psoas muscle area were determined at the level of L3 (cm2). RESULTS: Women had a higher subcutaneous (p < 0.001) and men had a higher visceral fat area, relative proportion of visceral fat area (p < 0.001), and psoas muscle index (p < 0.001). Logistic regression analysis showed an association between higher psoas muscle index and lower grade tumors [women: odds ratio (OR) 0.94, 95% confidence interval (CI) 0.89-0.99, p = 0.011; men: OR 0.97 (95% CI, 0.95-0.99, p = 0.012]. Univariate regression analysis demonstrated an association between psoas muscle index and overall survival (women: OR 1.41, 95% CI 1.03-1.93, p = 0.033; men: OR 1.62 (95% CI, 1.33-1.97, p < 0.001). In contrast, there were no associations between abdominal fat measurements and tumor size, grade, or survival. Also, there were no sex-specific differences in tumor size or tumor grades. CONCLUSIONS: A higher preoperative psoas muscle index was independently associated with overall survival in RCC patients, with a stronger association in men compared with women. In addition, the psoas muscle index showed an inverse association with tumor grade, whereby this association was slightly more pronounced in women than in men.

Can optimized model-based iterative reconstruction improve the contrast of liver lesions in CT?

BACKGROUND: Computed tomography is a standard imaging procedure for the detection of liver lesions, such as metastases, which can often be small and poorly contrasted, and therefore hard to detect. Advances in image reconstruction have shown promise in reducing image noise and improving low-contrast detectability. PURPOSE: To examine a novel, specialized, model-based iterative reconstruction (MBIR) technique for improved low-contrast liver lesion detection. MATERIAL AND METHODS: Patient images with reported poorly contrasted focal liver lesions were retrospectively reconstructed with the low-contrast attenuating algorithm (FIRST-LCD) from primary raw data. Liver-to-lesion contrast, signal-to-noise, and contrast-to-noise ratios for background and liver noise for each lesion were compared for all three FIRST-LCD presets with the established hybrid iterative reconstruction method (AIDR-3D). An additional visual conspicuity score was given by two experienced radiologists for each lesion. RESULTS: A total of 82 lesions in 57 examinations were included in the analysis. All three FIRST-LCD algorithms provided statistically significant increases in liver-to-lesion contrast, with FIRSTMILD showing the largest increase (40.47 HU in AIDR-3D; 45.84 HU in FIRSTMILD; P < 0.001). Substantial improvement was shown in contrast-to-noise metrics. Visual analysis of the lesions shows decreased lesion visibility with all FIRST methods in comparison to AIDR-3D, with FIRSTSTR showing the closest results (P < 0.001). CONCLUSION: Objective image metrics show promise for MBIR methods in improving the detectability of low-contrast liver lesions; however, subjective image quality may be perceived as inferior. Further improvements are necessary to enhance image quality and lesion detection.

What Does DALL-E 2 Know About Radiology?

Generative models, such as DALL-E 2 (OpenAI), could represent promising future tools for image generation, augmentation, and manipulation for artificial intelligence research in radiology, provided that these models have sufficient medical domain knowledge. Herein, we show that DALL-E 2 has learned relevant representations of x-ray images, with promising capabilities in terms of zero-shot text-to-image generation of new images, the continuation of an image beyond its original boundaries, and the removal of elements; however, its capabilities for the generation of images with pathological abnormalities (eg, tumors, fractures, and inflammation) or computed tomography, magnetic resonance imaging, or ultrasound images are still limited. The use of generative models for augmenting and generating radiological data thus seems feasible, even if the further fine-tuning and adaptation of these models to their respective domains are required first.

Deep Learning Detects Changes Indicative of Axial Spondyloarthritis at MRI of Sacroiliac Joints.

Background MRI is frequently used for early diagnosis of axial spondyloarthritis (axSpA). However, evaluation is time-consuming and requires profound expertise because noninflammatory degenerative changes can mimic axSpA, and early signs may therefore be missed. Deep neural networks could function as assistance for axSpA detection. Purpose To create a deep neural network to detect MRI changes in sacroiliac joints indicative of axSpA. Materials and Methods This retrospective multicenter study included MRI examinations of five cohorts of patients with clinical suspicion of axSpA collected at university and community hospitals between January 2006 and September 2020. Data from four cohorts were used as the training set, and data from one cohort as the external test set. Each MRI examination in the training and test sets was scored by six and seven raters, respectively, for inflammatory changes (bone marrow edema, enthesitis) and structural changes (erosions, sclerosis). A deep learning tool to detect changes indicative of axSpA was developed. First, a neural network to homogenize the images, then a classification network were trained. Performance was evaluated with use of area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. P < .05 was considered indicative of statistically significant difference. Results Overall, 593 patients (mean age, 37 years ± 11 [SD]; 302 women) were studied. Inflammatory and structural changes were found in 197 of 477 patients (41%) and 244 of 477 (51%), respectively, in the training set and 25 of 116 patients (22%) and 26 of 116 (22%) in the test set. The AUCs were 0.94 (95% CI: 0.84, 0.97) for all inflammatory changes, 0.88 (95% CI: 0.80, 0.95) for inflammatory changes fulfilling the Assessment of SpondyloArthritis international Society definition, and 0.89 (95% CI: 0.81, 0.96) for structural changes indicative of axSpA. Sensitivity and specificity on the external test set were 22 of 25 patients (88%) and 65 of 91 patients (71%), respectively, for inflammatory changes and 22 of 26 patients (85%) and 70 of 90 patients (78%) for structural changes. Conclusion Deep neural networks can detect inflammatory or structural changes to the sacroiliac joint indicative of axial spondyloarthritis at MRI. © RSNA, 2022 Online supplemental material is available for this article.

Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports.

MOTIVATION: The development of deep, bidirectional transformers such as Bidirectional Encoder Representations from Transformers (BERT) led to an outperformance of several Natural Language Processing (NLP) benchmarks. Especially in radiology, large amounts of free-text data are generated in daily clinical workflow. These report texts could be of particular use for the generation of labels in machine learning, especially for image classification. However, as report texts are mostly unstructured, advanced NLP methods are needed to enable accurate text classification. While neural networks can be used for this purpose, they must first be trained on large amounts of manually labelled data to achieve good results. In contrast, BERT models can be pre-trained on unlabelled data and then only require fine tuning on a small amount of manually labelled data to achieve even better results. RESULTS: Using BERT to identify the most important findings in intensive care chest radiograph reports, we achieve areas under the receiver operation characteristics curve of 0.98 for congestion, 0.97 for effusion, 0.97 for consolidation and 0.99 for pneumothorax, surpassing the accuracy of previous approaches with comparatively little annotation effort. Our approach could therefore help to improve information extraction from free-text medical reports. Availability and implementationWe make the source code for fine-tuning the BERT-models freely available at https://github.com/fast-raidiology/bert-for-radiology. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Deep learning for accurately recognizing common causes of shoulder pain on radiographs.

OBJECTIVE: Training a convolutional neural network (CNN) to detect the most common causes of shoulder pain on plain radiographs and to assess its potential value in serving as an assistive device to physicians. MATERIALS AND METHODS: We used a CNN of the ResNet-50 architecture which was trained on 2700 shoulder radiographs from clinical practice of multiple institutions. All radiographs were reviewed and labeled for six findings: proximal humeral fractures, joint dislocation, periarticular calcification, osteoarthritis, osteosynthesis, and joint endoprosthesis. The trained model was then evaluated on a separate test dataset, which was previously annotated by three independent expert radiologists. Both the training and the test datasets included radiographs of highly variable image quality to reflect the clinical situation and to foster robustness of the CNN. Performance of the model was evaluated using receiver operating characteristic (ROC) curves, the thereof derived AUC as well as sensitivity and specificity. RESULTS: The developed CNN demonstrated a high accuracy with an area under the curve (AUC) of 0.871 for detecting fractures, 0.896 for joint dislocation, 0.945 for osteoarthritis, and 0.800 for periarticular calcifications. It also detected osteosynthesis and endoprosthesis with near perfect accuracy (AUC 0.998 and 1.0, respectively). Sensitivity and specificity were 0.75 and 0.86 for fractures, 0.95 and 0.65 for joint dislocation, 0.90 and 0.86 for osteoarthrosis, and 0.60 and 0.89 for calcification. CONCLUSION: CNNs have the potential to serve as an assistive device by providing clinicians a means to prioritize worklists or providing additional safety in situations of increased workload.

CT-based quantification of short-term tissue shrinkage following hepatic microwave ablation in an in vivo porcine liver model.

BACKGROUND: Microwave ablation (MWA) is a minimally invasive treatment option for solid tumors and belongs to the local ablative therapeutic techniques, based on thermal tissue coagulation. So far there are mainly ex vivo studies that describe tissue shrinkage during MWA. PURPOSE: To characterize short-term volume changes of the ablated zone following hepatic MWA in an in vivo porcine liver model using contrast-enhanced computer tomography (CECT). MATERIAL AND METHODS: We performed multiple hepatic MWA with constant energy parameters in healthy, narcotized and laparotomized domestic pigs. The volumes of the ablated areas were calculated from venous phase CT scans, immediately after the ablation and in short-term courses of up to 2 h after MWA. RESULTS: In total, 19 thermally ablated areas in 10 porcine livers could be analyzed (n = 6 with two volume measurements during the measurement period and n = 13 with three measurements). Both groups showed a statistically significant but heterogeneous volume reduction of up to 12% (median 6%) of the ablated zones in CECT scans during the measurement period (P < 0.001 [n = 13] and P = 0.042 [n = 6]). However, the dimension and dynamics of volume changes were heterogenous both absolutely and relatively. CONCLUSION: We observed a significant short-term volume reduction of ablated liver tissue in vivo. This volume shrinkage must be considered in clinical practice for technically successful tumor treatment by MWA and therefore it should be further investigated in in vivo studies.

Immediate post-interventional contrast-enhanced computed tomography overestimates hepatic microwave ablation - an in vivo animal study.

Objectives: Contrast-enhanced computed tomography (CECT) is used to monitor technical success immediately after hepatic microwave ablation (MWA). However, it remains unclear, if CECT shows the exact extend of the thermal destruction zone, or if tissue changes such as peri-lesionary edema are depicted as well. The objective of this study was to correlate immediate post-interventional CECT with histological and macroscopic findings in hepatic MWA in porcine liver in vivo.Methods: Eleven MWA were performed in porcine liver in vivo with a microwave generator (928 MHz; energy input 24 kJ). CECT was performed post-interventionally. Livers were explanted and ablations were bisected immediately after ablation. Samples were histologically analyzed after vital staining (NADH-diaphorase). Ablation zones were histologically and macroscopically outlined. We correlated histologic findings, macroscopic images and CECT.Results: Three ablation zones were identified in histological and macroscopic findings. Only one ablation zone could be depicted in CECT. Close conformity was observed between histological and macroscopic findings. The ablation zone depicted in CECT overestimated the histological avital central zone and inner red zone (p < = .01). No differences were found between CECT and the histological outer red zone (p > .05).Conclusions: Immediate post-interventional CECT overestimated the clinically relevant zone of complete cell ablation after MWA in porcine liver in vivo. This entails the risk of incomplete tumor ablation and could lead to tumor recurrence.

Is lung density associated with severity of COVID-19?

PURPOSE: Emphysema and chronic obstructive lung disease were previously identified as major risk factors for severe disease progression in COVID-19. Computed tomography (CT)-based lung-density analysis offers a fast, reliable, and quantitative assessment of lung density. Therefore, we aimed to assess the benefit of CT-based lung density measurements to predict possible severe disease progression in COVID-19. MATERIAL AND METHODS: Thirty COVID-19-positive patients were included in this retrospective study. Lung density was quantified based on routinely acquired chest CTs. Presence of COVID-19 was confirmed by reverse transcription polymerase chain reaction (RT-PCR). Wilcoxon test was used to compare two groups of patients. A multivariate regression analysis, adjusted for age and sex, was employed to model the relative increase of risk for severe disease, depending on the measured densities. RESULTS: Intensive care unit (ICU) patients or patients requiring mechanical ventilation showed a lower proportion of medium- and low-density lung volume compared to patients on the normal ward, but a significantly larger volume of high-density lung volume (12.26 dl IQR 4.65 dl vs. 7.51 dl vs. IQR 5.39 dl, p = 0.039). In multivariate regression analysis, high-density lung volume was identified as a significant predictor of severe disease. CONCLUSIONS: The amount of high-density lung tissue showed a significant association with severe COVID-19, with odds ratios of 1.42 (95% CI: 1.09-2.00) and 1.37 (95% CI: 1.03-2.11) for requiring intensive care and mechanical ventilation, respectively. Acknowledging our small sample size as an important limitation; our study might thus suggest that high-density lung tissue could serve as a possible predictor of severe COVID-19.

Assessment of the extracellular volume fraction for the grading of clear cell renal cell carcinoma: first results and histopathological findings.

OBJECTIVES: To assess the potential of T1 mapping-based extracellular volume fraction (ECV) for the identification of higher grade clear cell renal cell carcinoma (cRCC), based on histopathology as the reference standard. METHODS: For this single-center, institutional review board-approved prospective study, 27 patients (17 men, median age 62 ± 12.4 years) with pathologic diagnosis of cRCC (nucleolar International Society of Urological Pathology (ISUP) grading) received abdominal MRI scans at 1.5 T using a modified Look-Locker inversion recovery (MOLLI) sequence between January 2017 and June 2018. Quantitative T1 values were measured at different time points (pre- and postcontrast agent administration) and quantification of the ECV was performed on MRI and histological sections (H&E staining). RESULTS: Reduction in T1 value after contrast agent administration and MR-derived ECV were reliable predictors for differentiating higher from lower grade cRCC. Postcontrast T1diff values (T1diff = T1 difference between the native and nephrogenic phase) and MR-derived ECV were significantly higher for higher grade cRCC (ISUP grades 3-4) compared with lower grade cRCC (ISUP grades 1-2) (p < 0.001). A cutoff value of 700 ms could distinguish higher grade from lower grade tumors with 100% (95% CI 0.69-1.00) sensitivity and 82% (95% CI 0.57-0.96) specificity. There was a positive and strong correlation between MR-derived ECV and histological ECV (p < 0.01, r = 0.88). Interobserver agreement for quantitative longitudinal relaxation times in the T1 maps was excellent. CONCLUSIONS: T1 mapping with ECV measurement could represent a novel in vivo biomarker for the classification of cRCC regarding their nucleolar grade, providing incremental diagnostic value as a quantitative MR marker. KEY POINTS: • Reduction in MRI T1 relaxation times after contrast agent administration and MR-derived extracellular volume fraction are useful parameters for grading of clear cell renal cell carcinoma (cRCC). • T1 differences between the native and the nephrogenic phase are higher for higher grade cRCC compared with lower grade cRCC and MRI-derived extracellular volume fraction (ECV) and histological ECV show a strong correlation. • T1 mapping with ECV measurement may be helpful for the noninvasive assessment of cRCC pathology, being a safe and feasible method, and it has potential to optimize individualized treatment options, e.g., in the decision of active surveillance.

Comparison of different 4D CT-Perfusion algorithms to visualize lesions after microwave ablation in an in vivo porcine model.

Background: Accurate lesion visualization after microwave ablation (MWA) remains a challenge. Computed tomography perfusion (CTP) has been proposed to improve visualization, but it was shown that different perfusion-models delivered different results on the same data set.Purpose: Comparison of different perfusion algorithms and identification of the algorithm enables for the best imaging of lesion after hepatic MWA.Materials and methods: 10 MWA with consecutive CTP were performed in healthy pigs. Parameter-maps were generated using a single-input-dual-compartment-model with Patlak's algorithm (PM), a dual-input-maximum-slope-model (DIMS), a dual-input-one-compartment-model (DIOC), a single-(SIDC) and dual-input-deconvolution-model (DIDC). Parameter-maps for hepatic arterial (AF) and portal venous blood flow (PF), mean transit time, hepatic blood volume (HBV) and capillary permeability were compared regarding the values of the normal liver tissue (NLT), lesion, contrast- and signal-to-noise ratios (SNR, CNR) and inter- and intrarater-reliability using the intraclass correlation coefficient, Bland-Altman plots and linear regression.Results: Perfusion values differed between algorithms with especially large fluctuations for the DIOC. A reliable differentiation of lesion margin appears feasible with parameter-maps of PF and HBV for most algorithms, except for the DIOC due to large fluctuations in PF. All algorithms allowed for a demarcation of the central necrotic zone based on hepatic AF and HBV. The DIDC showed the highest CNR and the best inter- and intrarater reliability.Conclusion: The DIDC appears to be the most feasible model to visualize margins and necrosis zones after microwave ablation, but due to high computational demand, a single input deconvolution algorithm might be preferable in clinical practice.

Comparative Analysis of Multimodal Large Language Model Performance on Clinical Vignette Questions.

This study compares 2 large language models and their performance vs that of competing open-source models.