RESUMEN
BACKGROUND: The medical coding of radiology reports is essential for a good quality of care and correct billing, but at the same time a complex and error-prone task. OBJECTIVE: To assess the performance of natural language processing (NLP) for ICD-10 coding of German radiology reports using fine tuning of suitable language models. MATERIAL AND METHODS: This retrospective study included all magnetic resonance imaging (MRI) radiology reports acquired at our institution between 2010 and 2020. The codes on discharge ICD-10 were matched to the corresponding reports to construct a dataset for multiclass classification. Fine tuning of GermanBERT and flanT5 was carried out on the total dataset (dstotal) containing 1035 different ICD-10 codes and 2 reduced subsets containing the 100 (ds100) and 50 (ds50) most frequent codes. The performance of the model was assessed using topk accuracy for kâ¯= 1, 3 and 5. In an ablation study both models were trained on the accompanying metadata and the radiology report alone. RESULTS: The total dataset consisted of 100,672 radiology reports, the reduced subsets ds100 of 68,103 and ds50 of 52,293 reports. The performance of the model increased when several of the best predictions of the model were taken into consideration, when the number of target classes was reduced and the metadata were combined with the report. The flanT5 outperformed GermanBERT across all datasets and metrics and was is suited as a medical coding assistant, achieving a top 3 accuracy of nearly 70% in the real-world dataset dstotal. CONCLUSION: Finely tuned language models can reliably predict ICD-10 codes of German magnetic resonance imaging (MRI) radiology reports across various settings. As a coding assistant flanT5 can guide medical coders to make informed decisions and potentially reduce the workload.
RESUMEN
CT protocols that diagnose COVID-19 vary in regard to the associated radiation exposure and the desired image quality (IQ). This study aims to evaluate CT protocols of hospitals participating in the RACOON (Radiological Cooperative Network) project, consolidating CT protocols to provide recommendations and strategies for future pandemics. In this retrospective study, CT acquisitions of COVID-19 patients scanned between March 2020 and October 2020 (RACOON phase 1) were included, and all non-contrast protocols were evaluated. For this purpose, CT protocol parameters, IQ ratings, radiation exposure (CTDIvol), and central patient diameters were sampled. Eventually, the data from 14 sites and 534 CT acquisitions were analyzed. IQ was rated good for 81% of the evaluated examinations. Motion, beam-hardening artefacts, or image noise were reasons for a suboptimal IQ. The tube potential ranged between 80 and 140 kVp, with the majority between 100 and 120 kVp. CTDIvol was 3.7 ± 3.4 mGy. Most healthcare facilities included did not have a specific non-contrast CT protocol. Furthermore, CT protocols for chest imaging varied in their settings and radiation exposure. In future, it will be necessary to make recommendations regarding the required IQ and protocol parameters for the majority of CT scanners to enable comparable IQ as well as radiation exposure for different sites but identical diagnostic questions.
RESUMEN
OBJECTIVES: To assess the quality of simplified radiology reports generated with the large language model (LLM) ChatGPT and to discuss challenges and chances of ChatGPT-like LLMs for medical text simplification. METHODS: In this exploratory case study, a radiologist created three fictitious radiology reports which we simplified by prompting ChatGPT with "Explain this medical report to a child using simple language." In a questionnaire, we tasked 15 radiologists to rate the quality of the simplified radiology reports with respect to their factual correctness, completeness, and potential harm for patients. We used Likert scale analysis and inductive free-text categorization to assess the quality of the simplified reports. RESULTS: Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient. Nevertheless, instances of incorrect statements, missed relevant medical information, and potentially harmful passages were reported. CONCLUSION: While we see a need for further adaption to the medical field, the initial insights of this study indicate a tremendous potential in using LLMs like ChatGPT to improve patient-centered care in radiology and other medical domains. CLINICAL RELEVANCE STATEMENT: Patients have started to use ChatGPT to simplify and explain their medical reports, which is expected to affect patient-doctor interaction. This phenomenon raises several opportunities and challenges for clinical routine. KEY POINTS: ⢠Patients have started to use ChatGPT to simplify their medical reports, but their quality was unknown. ⢠In a questionnaire, most participating radiologists overall asserted good quality to radiology reports simplified with ChatGPT. However, they also highlighted a notable presence of errors, potentially leading patients to draw harmful conclusions. ⢠Large language models such as ChatGPT have vast potential to enhance patient-centered care in radiology and other medical domains. To realize this potential while minimizing harm, they need supervision by medical experts and adaption to the medical field.
RESUMEN
Objectives: The aim of this study was to investigate an integrated diagnostics approach for prediction of the source of aldosterone overproduction in primary hyperaldosteronism (PA). Methods: 269 patients from the prospective German Conn Registry with PA were included in this study. After segmentation of adrenal glands in native CT images, radiomic features were calculated. The study population consisted of a training (n = 215) and a validation (n = 54) cohort. The k = 25 best radiomic features, selected using maximum-relevance minimum-redundancy (MRMR) feature selection, were used to train a baseline random forest model to predict the result of AVS from imaging alone. In a second step, clinical parameters were integrated. Model performance was assessed via area under the receiver operating characteristic curve (ROC AUC). Permutation feature importance was used to assess the predictive value of selected features. Results: Radiomics features alone allowed only for moderate discrimination of the location of aldosterone overproduction with a ROC AUC of 0.57 for unilateral left (UL), 0.61 for unilateral right (UR), and 0.50 for bilateral (BI) aldosterone overproduction (total 0.56, 95% CI: 0.45-0.65). Integration of clinical parameters into the model substantially improved ROC AUC values (0.61 UL, 0.68 UR, and 0.73 for BI, total 0.67, 95% CI: 0.57-0.77). According to permutation feature importance, lowest potassium value at baseline and saline infusion test (SIT) were the two most important features. Conclusion: Integration of clinical parameters into a radiomics machine learning model improves prediction of the source of aldosterone overproduction and subtyping in patients with PA.
Asunto(s)
Aldosterona , Hiperaldosteronismo , Humanos , Estudios Prospectivos , Aprendizaje Automático , Hiperaldosteronismo/diagnóstico por imagen , Tomografía Computarizada por Rayos XRESUMEN
OBJECTIVES: Optimizing a machine learning (ML) pipeline for radiomics analysis involves numerous choices in data set composition, preprocessing, and model selection. Objective identification of the optimal setup is complicated by correlated features, interdependency structures, and a multitude of available ML algorithms. Therefore, we present a radiomics-based benchmarking framework to optimize a comprehensive ML pipeline for the prediction of overall survival. This study is conducted on an image set of patients with hepatic metastases of colorectal cancer, for which radiomics features of the whole liver and of metastases from computed tomography images were calculated. A mixed model approach was used to find the optimal pipeline configuration and to identify the added prognostic value of radiomics features. MATERIALS AND METHODS: In this study, a large-scale ML benchmark pipeline consisting of preprocessing, feature selection, dimensionality reduction, hyperparameter optimization, and training of different models was developed for radiomics-based survival analysis. Portal-venous computed tomography imaging data from a previous prospective randomized trial evaluating radioembolization of liver metastases of colorectal cancer were quantitatively accessible through a radiomics approach. One thousand two hundred eighteen radiomics features of hepatic metastases and the whole liver were calculated, and 19 clinical parameters (age, sex, laboratory values, and treatment) were available for each patient. Three ML algorithms-a regression model with elastic net regularization (glmnet), a random survival forest (RSF), and a gradient tree-boosting technique (xgboost)-were evaluated for 5 combinations of clinical data, tumor radiomics, and whole-liver features. Hyperparameter optimization and model evaluation were optimized toward the performance metric integrated Brier score via nested cross-validation. To address dependency structures in the benchmark setup, a mixed-model approach was developed to compare ML and data configurations and to identify the best-performing model. RESULTS: Within our radiomics-based benchmark experiment, 60 ML pipeline variations were evaluated on clinical data and radiomics features from 491 patients. Descriptive analysis of the benchmark results showed a preference for RSF-based pipelines, especially for the combination of clinical data with radiomics features. This observation was supported by the quantitative analysis via a linear mixed model approach, computed to differentiate the effect of data sets and pipeline configurations on the resulting performance. This revealed the RSF pipelines to consistently perform similar or better than glmnet and xgboost. Further, for the RSF, there was no significantly better-performing pipeline composition regarding the sort of preprocessing or hyperparameter optimization. CONCLUSIONS: Our study introduces a benchmark framework for radiomics-based survival analysis, aimed at identifying the optimal settings with respect to different radiomics data sources and various ML pipeline variations, including preprocessing techniques and learning algorithms. A suitable analysis tool for the benchmark results is provided via a mixed model approach, which showed for our study on patients with intrahepatic liver metastases, that radiomics features captured the patients' clinical situation in a manner comparable to the provided information solely from clinical parameters. However, we did not observe a relevant additional prognostic value obtained by these radiomics features.
Asunto(s)
Neoplasias Colorrectales , Neoplasias Hepáticas , Humanos , Benchmarking , Neoplasias Hepáticas/diagnóstico por imagen , Tomografía Computarizada por Rayos X , Aprendizaje Automático , Análisis de Supervivencia , Neoplasias Colorrectales/diagnóstico por imagen , Estudios RetrospectivosRESUMEN
BACKGROUND: Deep learning is a promising technique to improve radiological age assessment. However, expensive manual annotation by experts poses a bottleneck for creating large datasets to appropriately train deep neural networks. We propose an object detection approach to automatically annotate the medial clavicular epiphyseal cartilages in computed tomography (CT) scans. METHODS: The sternoclavicular joints were selected as structure-of-interest (SOI) in chest CT scans and served as an easy-to-identify proxy for the actual medial clavicular epiphyseal cartilages. CT slices containing the SOI were manually annotated with bounding boxes around the SOI. All slices in the training set were used to train the object detection network RetinaNet. Afterwards, the network was applied individually to all slices of the test scans for SOI detection. Bounding box and slice position of the detection with the highest classification score were used as the location estimate for the medial clavicular epiphyseal cartilages inside the CT scan. RESULTS: From 100 CT scans of 82 patients, 29,656 slices were used for training and 30,846 slices from 110 CT scans of 110 different patients for testing the object detection network. The location estimate from the deep learning approach for the SOI was in a correct slice in 97/110 (88%), misplaced by one slice in 5/110 (5%), and missing in 8/110 (7%) test scans. No estimate was misplaced by more than one slice. CONCLUSIONS: We demonstrated a robust automated approach for annotating the medial clavicular epiphyseal cartilages. This enables training and testing of deep neural networks for age assessment.
Asunto(s)
Aprendizaje Profundo , Placa de Crecimiento , Humanos , Placa de Crecimiento/diagnóstico por imagen , Tomografía Computarizada por Rayos X/métodos , Redes Neurales de la Computación , Clavícula/diagnóstico por imagenRESUMEN
(1) Background: CT perfusion (CTP) is used to quantify cerebral hypoperfusion in acute ischemic stroke. Conventional attenuation curve analysis is not standardized and might require input from expert users, hampering clinical application. This study aims to bypass conventional tracer-kinetic analysis with an end-to-end deep learning model to directly categorize patients by stroke core volume from raw, slice-reduced CTP data. (2) Methods: In this retrospective analysis, we included patients with acute ischemic stroke due to proximal occlusion of the anterior circulation who underwent CTP imaging. A novel convolutional neural network was implemented to extract spatial and temporal features from time-resolved imaging data. In a classification task, the network categorized patients into small or large core. In ten-fold cross-validation, the network was repeatedly trained, evaluated, and tested, using the area under the receiver operating characteristic curve (ROC-AUC). A final model was created in an ensemble approach and independently validated on an external dataset. (3) Results: 217 patients were included in the training cohort and 23 patients in the independent test cohort. Median core volume was 32.4 mL and was used as threshold value for the binary classification task. Model performance yielded a mean (SD) ROC-AUC of 0.72 (0.10) for the test folds. External independent validation resulted in an ensembled mean ROC-AUC of 0.61. (4) Conclusions: In this proof-of-concept study, the proposed end-to-end deep learning approach bypasses conventional perfusion analysis and allows to predict dichotomized infarction core volume solely from slice-reduced CTP images without underlying tracer kinetic assumptions. Further studies can easily extend to additional clinically relevant endpoints.
RESUMEN
BACKGROUND: Yttrium-90 radioembolization (RE) plays an important role in the treatment of liver malignancies. Optimal patient selection is crucial for an effective and safe treatment. In this study, we aim to validate the prognostic performance of a previously established random survival forest (RSF) with an external validation cohort from a different national center. Furthermore, we compare outcome prediction models with different established metrics. METHODS: A previously established RSF model, trained on a consecutive cohort of 366 patients who had received RE due to primary or secondary liver tumor at a national center (center 1), was used to predict the outcome of an independent consecutive cohort of 202 patients from a different national center (center 2) and vice versa. Prognostic performance was evaluated using the concordance index (C-index) and the integrated Brier score (IBS). The prognostic importance of designated baseline parameters was measured with the minimal depth concept, and the influence on the predicted outcome was analyzed with accumulated local effects plots. RSF values were compared to conventional cox proportional hazards models in terms of C-index and IBS. RESULTS: The established RSF model achieved a C-index of 0.67 for center 2, comparable to the results obtained for center 1, which it was trained on (0.66). The RSF model trained on center 2 achieved a C-index of 0.68 on center 2 data and 0.66 on center 1 data. CPH models showed comparable results on both cohorts, with C-index ranging from 0.68 to 0.72. IBS validation showed more differentiated results depending on which cohort was trained on and which cohort was predicted (range: 0.08 to 0.20). Baseline cholinesterase was the most important variable for survival prediction. CONCLUSION: The previously developed predictive RSF model was successfully validated with an independent external cohort. C-index and IBS are suitable metrics to compare outcome prediction models, with IBS showing more differentiated results. The findings corroborate that survival after RE is critically determined by functional hepatic reserve and thus baseline liver function should play a key role in patient selection.
RESUMEN
BACKGROUND: To visualize and assess brain metastases on magnetic resonance imaging, radiologists face an ever-increasing pressure to perform faster and more efficiently. The usage of maximum intensity projections (MIPs) of contrast-enhanced T1-weighed (T1ce) magnetization-prepared rapid acquisition with gradient echo (MP-RAGE) images proposes to increase reading efficiency by increasing lesion conspicuity while reducing in the number of images to be reviewed. AIM: To assess if MIPs save reading time and achieve the same level of diagnostic accuracy as standard 1 mm T1ce images for the detection of brain metastases. METHODS: Forty-four patients were included in this retrospective study. Axial reformations of T1ce MP-RAGE (TR/TE = 2300/2.25 ms, resolution = 1 mm3 ) images were analyzed and post-processed into 5 and 10 mm MIPs. Two readers evaluated the randomly assorted images and recorded reading time. Reading time differences were analyzed using the Wilcoxon test, and inter-reader statistics were performed using Bland-Altman plots. RESULTS: About 22.5 61.2 s/study and 43.8 ± 159.9 s/study were saved using 5 and 10 mm MIPs, respectively. Combined average sensitivity was 92.0% for 5 mm MIPs and 86.3% for 10 mm MIPs compared to standard 1 mm axial slices, with an average rate of 0.98 and 0.57 false positives per study, respectively CONCLUSION: While 5 mm and 10 mm T1ce MP-RAGE MIPs showed a clinical benefit in reducing reading times for evaluation of brain metastases, they should be used in conjunction with standard 1 mm images for best sensitivity and specificity, a practice which possibly annuls their benefit.
Asunto(s)
Neoplasias Encefálicas/diagnóstico , Encéfalo/diagnóstico por imagen , Imagen de Difusión por Resonancia Magnética/métodos , Imagenología Tridimensional , Adulto , Anciano , Neoplasias Encefálicas/secundario , Medios de Contraste/administración & dosificación , Femenino , Humanos , Masculino , Persona de Mediana Edad , Estudios Retrospectivos , Sensibilidad y EspecificidadRESUMEN
Tracer-kinetic analysis of dynamic contrast-enhanced magnetic resonance imaging data is commonly performed with the well-known Tofts model and nonlinear least squares (NLLS) regression. This approach yields point estimates of model parameters, uncertainty of these estimates can be assessed e.g. by an additional bootstrapping analysis. Here, we present a Bayesian probabilistic modeling approach for tracer-kinetic analysis with a Tofts model, which yields posterior probability distributions of perfusion parameters and therefore promises a robust and information-enriched alternative based on a framework of probability distributions. In this manuscript, we use the quantitative imaging biomarkers alliance (QIBA) Tofts phantom to evaluate the Bayesian tofts model (BTM) against a bootstrapped NLLS approach. Furthermore, we demonstrate how Bayesian posterior probability distributions can be employed to assess treatment response in a breast cancer DCE-MRI dataset using Cohen's d. Accuracy and precision of the BTM posterior distributions were validated and found to be in good agreement with the NLLS approaches, and assessment of therapy response with respect to uncertainty in parameter estimates was found to be excellent. In conclusion, the Bayesian modeling approach provides an elegant means to determine uncertainty via posterior distributions within a single step and provides honest information about changes in parameter estimates.