RESUMO
OBJECTIVES: To assess the capabilities of large language models (LLMs), including Open AI (GPT-4.0) and Microsoft Bing (GPT-4), in generating structured reports, the Breast Imaging Reporting and Data System (BI-RADS) categories, and management recommendations from free-text breast ultrasound reports. MATERIALS AND METHODS: In this retrospective study, 100 free-text breast ultrasound reports from patients who underwent surgery between January and May 2023 were gathered. The capabilities of Open AI (GPT-4.0) and Microsoft Bing (GPT-4) to convert these unstructured reports into structured ultrasound reports were studied. The quality of structured reports, BI-RADS categories, and management recommendations generated by GPT-4.0 and Bing were evaluated by senior radiologists based on the guidelines. RESULTS: Open AI (GPT-4.0) was better than Microsoft Bing (GPT-4) in terms of performance in generating structured reports (88% vs. 55%; p < 0.001), giving correct BI-RADS categories (54% vs. 47%; p = 0.013) and providing reasonable management recommendations (81% vs. 63%; p < 0.001). As the ability to predict benign and malignant characteristics, GPT-4.0 performed significantly better than Bing (AUC, 0.9317 vs. 0.8177; p < 0.001), while both performed significantly inferior to senior radiologists (AUC, 0.9763; both p < 0.001). CONCLUSION: This study highlights the potential of LLMs, specifically Open AI (GPT-4.0), in converting unstructured breast ultrasound reports into structured ones, offering accurate diagnoses and providing reasonable recommendations.
RESUMO
Computer-aided diagnosis (CAD) for thyroid nodules has been studied for years, yet there are still reliability and interpretability challenges due to the lack of clinically-relevant evidence. To address this issue, inspired by Thyroid Imaging Reporting and Data System (TI-RADS), we propose a novel interpretable two-branch bi-coordinate network based on multi-grained domain knowledge. First, we transform the two types of domain knowledge provided by TI-RADS, namely region-based and boundary-based knowledge, into labels at multi-grained levels: coarse-grained classification labels, and fine-grained region segmentation masks and boundary localization vectors. We combine these two labels to form the Multi-grained Domain Knowledge Representation (MG-DKR) of TI-RADS. Then we design a Two-branch Bi-coordinate network (TB2C-net) which utilizes two branches to predict MG-DKR from both Cartesian and polar images, and uses an attention-based integration module to integrate the features of the two branches for benign-malignant classification. We validated our method on a large cohort containing 3245 patients (with 3558 nodules and 6466 ultrasound images). Results show that our method achieves competitive performance with AUC of 0.93 and ACC of 0.87 compared with other state-of-the-art methods. Ablation experiment results demonstrate the effectiveness of the TB2C-net and MG-DKR, and the knowledge attention map from the integration module provides the interpretability for benign-malignant classification.
RESUMO
PURPOSE: The importance of structured radiology reports has been fully recognized, as they facilitate efficient data extraction and promote collaboration among healthcare professionals. Our purpose is to assess the accuracy and reproducibility of ChatGPT, a large language model, in generating structured thyroid ultrasound reports. METHODS: This is a retrospective study that includes 184 nodules in 136 thyroid ultrasound reports from 136 patients. ChatGPT-3.5 and ChatGPT-4.0 were used to structure the reports based on ACR-TIRADS guidelines. Two radiologists evaluated the responses for quality, nodule categorization accuracy, and management recommendations. Each text was submitted twice to assess the consistency of the nodule classification and management recommendations. RESULTS: On 136 ultrasound reports from 136 patients (mean age, 52 years ± 12 [SD]; 61 male), ChatGPT-3.5 generated 202 satisfactory structured reports, while ChatGPT-4.0 only produced 69 satisfactory structured reports (74.3 % vs. 25.4 %, odds ratio (OR) = 8.490, 95 %CI: 5.775-12.481, p < 0.001). ChatGPT-4.0 outperformed ChatGPT-3.5 in categorizing thyroid nodules, with an accuracy of 69.3 % compared to 34.5 % (OR = 4.282, 95 %CI: 3.145-5.831, p < 0.001). ChatGPT-4.0 also provided more comprehensive or correct management recommendations than ChatGPT-3.5 (OR = 1.791, 95 %CI: 1.297-2.473, p < 0.001). Finally, ChatGPT-4.0 exhibits higher consistency in categorizing nodules compared to ChatGPT-3.5 (ICC = 0.732 vs. ICC = 0.429), and both exhibited moderate consistency in management recommendations (ICC = 0.549 vs ICC = 0.575). CONCLUSIONS: Our study demonstrates the potential of ChatGPT in transforming free-text thyroid ultrasound reports into structured formats. ChatGPT-3.5 excels in generating structured reports, while ChatGPT-4.0 shows superior accuracy in nodule categorization and management recommendations.
Assuntos
Sistemas de Informação em Radiologia , Nódulo da Glândula Tireoide , Ultrassonografia , Humanos , Pessoa de Meia-Idade , Masculino , Feminino , Ultrassonografia/métodos , Nódulo da Glândula Tireoide/diagnóstico por imagem , Reprodutibilidade dos Testes , Estudos Retrospectivos , Processamento de Linguagem Natural , Glândula Tireoide/diagnóstico por imagem , AdultoRESUMO
BACKGROUND: Benign prostatic hyperplasia (BPH) is a common condition, yet it is challenging for the average BPH patient to find credible and accurate information about BPH. Our goal is to evaluate and compare the accuracy and reproducibility of large language models (LLMs), including ChatGPT-3.5, ChatGPT-4, and the New Bing Chat in responding to a BPH frequently asked questions (FAQs) questionnaire. METHODS: A total of 45 questions related to BPH were categorized into basic and professional knowledge. Three LLM-ChatGPT-3.5, ChatGPT-4, and New Bing Chat-were utilized to generate responses to these questions. Responses were graded as comprehensive, correct but inadequate, mixed with incorrect/outdated data, or completely incorrect. Reproducibility was assessed by generating two responses for each question. All responses were reviewed and judged by experienced urologists. RESULTS: All three LLMs exhibited high accuracy in generating responses to questions, with accuracy rates ranging from 86.7% to 100%. However, there was no statistically significant difference in response accuracy among the three (p > 0.017 for all comparisons). Additionally, the accuracy of the LLMs' responses to the basic knowledge questions was roughly equivalent to that of the specialized knowledge questions, showing a difference of less than 3.5% (GPT-3.5: 90% vs. 86.7%; GPT-4: 96.7% vs. 95.6%; New Bing: 96.7% vs. 93.3%). Furthermore, all three LLMs demonstrated high reproducibility, with rates ranging from 93.3% to 97.8%. CONCLUSIONS: ChatGPT-3.5, ChatGPT-4, and New Bing Chat offer accurate and reproducible responses to BPH-related questions, establishing them as valuable resources for enhancing health literacy and supporting BPH patients in conjunction with healthcare professionals.
Assuntos
Hiperplasia Prostática , Humanos , Hiperplasia Prostática/diagnóstico , Masculino , Reprodutibilidade dos Testes , Inquéritos e Questionários , Idioma , Educação de Pacientes como Assunto/métodosRESUMO
OBJECTIVE: This study aimed to investigate the predictive factors as well as the time and age course of recurrence/persistence in a large cohort of postoperative patients with papillary thyroid carcinoma (PTC) based on the long-term ultrasonography (US) follow-up data. METHODS: Between January 2007 and December 2016, 3106 patients underwent surgery for PTC and at least two postoperative US follow-up examination over more than three years. Tumor recurrence/persistence was confirmed based on the follow-up US data and histopathological results. Univariate and multivariate analyses were performed to evaluate the predictive factors of tumor recurrence/persistence. Kaplan-Meier survival analysis was used to evaluate the recurrence-/persistence-free survival curve based on the US results. RESULTS: A total of 321(10.3%) patients developed tumor recurrence/persistence during 54.3 months of mean follow-up (range 36-135 months), including 268(83.5%) cases of lymph node recurrence/persistence, 37 (11.5%) cases of non-lymph node recurrence/persistence, and 16(5%) cases of both types. Recurrence/persistence was observed using US examination at a mean interval of 23.6 ± 21.6 months (range 1-135 months) after surgery and peak incidence was observed 1-2 years after initial treatment. Younger (20-30 years old) and older (70-80 years old) patients had a higher proportion of tumor recurrence/persistence. Multifocality, advanced T and advanced N stages were independent risk factors of tumor recurrence/persistence. CONCLUSION: Tumor recurrence/persistence of PTC usually occurs during the early postoperative period. For patients with multifocal cancer, advanced T and N stage, the US surveillance examination should be cautiously performed, especially in younger and older patients.
Assuntos
Carcinoma Papilar , Neoplasias da Glândula Tireoide , Adulto , Idoso , Idoso de 80 Anos ou mais , Carcinoma Papilar/diagnóstico por imagem , Carcinoma Papilar/cirurgia , Seguimentos , Humanos , Recidiva Local de Neoplasia/epidemiologia , Estudos Retrospectivos , Câncer Papilífero da Tireoide/diagnóstico por imagem , Câncer Papilífero da Tireoide/cirurgia , Neoplasias da Glândula Tireoide/diagnóstico por imagem , Neoplasias da Glândula Tireoide/cirurgia , Tireoidectomia , Ultrassonografia , Adulto JovemRESUMO
OBJECTIVE: The aim of this study is to develop a model using Deep Neural Network (DNN) to diagnose thyroid nodules in patients with Hashimoto's Thyroiditis. METHODS: In this retrospective study, we included 2,932 patients with thyroid nodules who underwent thyroid ultrasonogram in our hospital from January 2017 to August 2019. 80% of them were included as training set and 20% as test set. Nodules suspected for malignancy underwent FNA or surgery for pathological results. Two DNN models were trained to diagnose thyroid nodules, and we chose the one with better performance. The features of nodules as well as parenchyma around nodules will be learned by the model to achieve better performance under diffused parenchyma. 10-fold cross-validation and an independent test set were used to evaluate the performance of the algorithm. The performance of the model was compared with that of the three groups of radiologists with clinical experience of <5 years, 5-10 years, >10 years respectively. RESULTS: In total, 9,127 images were collected from 2,932 patients with 7,301 images for the training set and 1,806 for the test set. 56% of the patients enrolled had Hashimoto's Thyroiditis. The model achieved an AUC of 0.924 for distinguishing malignant and benign nodules in the test set. It showed similar performance under diffused thyroid parenchyma and normal parenchyma with sensitivity of 0.881 versus 0.871 (p = 0.938) and specificity of 0.846 versus 0.822 (p = 0.178). In patients with HT, the model achieved an AUC of 0.924 to differentiate malignant and benign nodules which was significantly higher than that of the three groups of radiologists (AUC = 0.824, 0.857, 0.863 respectively, p < 0.05). CONCLUSION: The model showed high performance in diagnosing thyroid nodules under both normal and diffused parenchyma. In patients with Hashimoto's Thyroiditis, the model showed a better performance compared to radiologists with various years of experience.