Evaluating ChatGPT-4V in chest CT diagnostics: a critical image interpretation assessment.

PURPOSE: To assess the diagnostic accuracy of ChatGPT-4V in interpreting a set of four chest CT slices for each case of COVID-19, non-small cell lung cancer (NSCLC), and control cases, thereby evaluating its potential as an AI tool in radiological diagnostics. MATERIALS AND METHODS: In this retrospective study, 60 CT scans from The Cancer Imaging Archive, covering COVID-19, NSCLC, and control cases were analyzed using ChatGPT-4V. A radiologist selected four CT slices from each scan for evaluation. ChatGPT-4V's interpretations were compared against the gold standard diagnoses and assessed by two radiologists. Statistical analyses focused on accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), along with an examination of the impact of pathology location and lobe involvement. RESULTS: ChatGPT-4V showed an overall diagnostic accuracy of 56.76%. For NSCLC, sensitivity was 27.27% and specificity was 60.47%. In COVID-19 detection, sensitivity was 13.64% and specificity of 64.29%. For control cases, the sensitivity was 31.82%, with a specificity of 95.24%. The highest sensitivity (83.33%) was observed in cases involving all lung lobes. The chi-squared statistical analysis indicated significant differences in Sensitivity across categories and in relation to the location and lobar involvement of pathologies. CONCLUSION: ChatGPT-4V demonstrated variable diagnostic performance in chest CT interpretation, with notable proficiency in specific scenarios. This underscores the challenges of cross-modal AI models like ChatGPT-4V in radiology, pointing toward significant areas for improvement to ensure dependability. The study emphasizes the importance of enhancing these models for broader, more reliable medical use.

Shaping a suitable EU HTA dossier template: why the German template is not fit for purpose.

From 2025, Health Technology Developers (HTDs) have to submit EU HTA dossiers. The joint clinical assessment (JCA) aims to streamline HTA processes and access to medicinal products across Europe. Currently, German HTA bodies IQWiG and G-BA actively shape the JCA methodology. Here we examine if German HTA dossier requirements are suitable for the JCA. We compare the number of safety endpoint and subgroup analyses in German dossiers with analyses considered in IQWIG's benefit assessment and evaluate if these analyses were considered by the G-BA. We further investigated how the number of analyses was affected by the latest change in the German dossier template. With the current template, HTDs report in median 2.6 times more analyses on adverse events (AE) and 1.1 times more subgroup categories than in the previous template. IQWiG does not consider 33% of AE analyses and 73% of the subgroup categories presented by the HTD under the current template. G-BA considered the same AE as IQWiG in 76% of cases. Subgroups were uncommented by G-BA in most cases, independent of the template (previous: 93%, current 85%) and unconsidered in the conclusion on additional benefit (previous: 77%, current 69%). Thus, changes in the dossier template drastically increased HTD workload, but additional analyses seem unconsidered by the HTA bodies. With a broader scope in JCA, this effect could be amplified. To mitigate duplicative efforts and ensure prompt availability of medicinal products as envisioned by the HTAR, we suggest well-chosen and precise dossier requirements, early consultations, and early HTD engagement.

Faster Elbow MRI with Deep Learning Reconstruction-Assessment of Image Quality, Diagnostic Confidence, and Anatomy Visualization Compared to Standard Imaging.

OBJECTIVE: The objective of this study was to evaluate a deep learning (DL) reconstruction for turbo spin echo (TSE) sequences of the elbow regarding image quality and visualization of anatomy. MATERIALS AND METHODS: Between October 2020 and June 2021, seventeen participants (eight patients, nine healthy subjects; mean age: 43 ± 16 (20-70) years, eight men) were prospectively included in this study. Each patient underwent two examinations: standard MRI, including TSE sequences reconstructed with a generalized autocalibrating partial parallel acquisition reconstruction (TSESTD), and prospectively undersampled TSE sequences reconstructed with a DL reconstruction (TSEDL). Two radiologists evaluated the images concerning image quality, noise, edge sharpness, artifacts, diagnostic confidence, and delineation of anatomical structures using a 5-point Likert scale, and rated the images concerning the detection of common pathologies. RESULTS: Image quality was significantly improved in TSEDL (mean 4.35, IQR 4-5) compared to TSESTD (mean 3.76, IQR 3-4, p = 0.008). Moreover, TSEDL showed decreased noise (mean 4.29, IQR 3.5-5) compared to TSESTD (mean 3.35, IQR 3-4, p = 0.004). Ratings for delineation of anatomical structures, artifacts, edge sharpness, and diagnostic confidence did not differ significantly between TSEDL and TSESTD (p > 0.05). Inter-reader agreement was substantial to almost perfect (κ = 0.628-0.904). No difference was found concerning the detection of pathologies between the readers and between TSEDL and TSESTD. Using DL, the acquisition time could be reduced by more than 35% compared to TSESTD. CONCLUSION: TSEDL provided improved image quality and decreased noise while receiving equal ratings for edge sharpness, artifacts, delineation of anatomical structures, diagnostic confidence, and detection of pathologies compared to TSESTD. Providing more than a 35% reduction of acquisition time, TSEDL may be clinically relevant for elbow imaging due to increased patient comfort and higher patient throughput.

Dual-Energy CT-Based Bone Marrow Imaging in Multiple Myeloma: Assessment of Focal Lesions in Relation to Disease Status and MRI Findings.

RATIONALE AND OBJECTIVES: To assess focal multiple myeloma bone lesions via dual-energy CT-based virtual noncalcium (VNCa) bone marrow imaging in relation to the overall hematological disease status and MRI findings. MATERIALS AND METHODS: We retrospectively evaluated 103 focal osteolytic lesions of the axial skeleton in VNCa bone marrow images of 32 patients. Region of interest-based attenuation measurements were correlated with T1w signal intensity and apparent diffusion coefficient (ADC). Results were compared between patients in active and inactive disease. Receiver operating characteristic analysis was performed to determine a cut-off value of VNCa attenuation for differentiation between the two groups. Standard of reference was the overall disease status according to International Myeloma Working Group response criteria. RESULTS: Mean attenuation difference between lesions and background bone marrow was significantly lower in inactive disease (16 HU, SD 30) compared to active disease (35 HU, SD 29). VNCa attenuation measurement allowed for differentiation between active and inactive disease with a sensitivity of 92% and a specificity of 58% at a cut-off value of -21 HU. VNCa attenuation was negatively correlated to T1w signal intensity (Spearman's ρ -0.617, p < 0.001) and positively correlated to ADC (Spearman's ρ 0.521, p < 0.001). CONCLUSION: Quantitative assessment of attenuation of focal osteolytic lesions in VNCa bone marrow images allows differentiation between overall active and inactive disease with higher attenuation signifying an increasing likelihood of active disease. This is supported by a significant positive correlation between the attenuation and the ADC, as well as a corresponding inverse correlation to T1w signal intensity.

Towards compatibility of EUnetHTA JCA methodology and German HTA: a systematic comparison and recommendations from an industry perspective.

OBJECTIVE: The transferability of the EU joint clinical assessment (JCA) reports for pharmaceuticals for the German benefit assessment was evaluated by systematically comparing EU JCA and German clinical assessments (CA) based on established assessment elements for HTA and assessing the potential impact of differences on Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA) ability to derive the therapeutic added value. METHODS: Identification of all pharmaceuticals undergoing both, EU JCA and German CA between January 2016-June 2020. Qualitative review and data extraction from the assessments, assessment of methodological differences using a hierarchical model. Recommendations for harmonisation were developed and consented with pharmaceutical industry stakeholders. RESULTS: Differences with potentially major impact: (1) View on differing treatment algorithms and definition of corresponding subpopulations/respective comparators. (2) Clinical relevance of surrogate/intermediate endpoints. Inclusion of different/surrogate morbidity endpoints resulting in different relative effectiveness conclusions. (3) Tolerance of study interventions not used according to marketing authorisation. (4) Different operationalisation and/or weighting of individual safety endpoints leading to differing relative safety conclusions. Differences with potentially minor impact: (1) Disagreement in risk of bias assessment for overall survival and its robustness against study limitations. (2) Use of patient-reported outcome symptom scales as measurements for health-related quality of life instruments. CONCLUSION: While many synergies between EU JCA and German CA exist, we identified several aspects in HTA methodology that would benefit of harmonisation and ensure the transferability of future EU JCA to the German HTA process without duplicated evaluation requirements. For those, a set of recommendations was developed.

Postponing elective hospitalizations for pre-admission MRSA screening and decolonization. A study evaluating eligibility and acceptance among patients of a German university hospital.

There is evidence that pre-admission screening and decolonization (PreASD) of MRSA can reduce costs in elective surgical patients. It is not known whether this strategy could also be successfully applied to general medical patients of a tertiary referral hospital with multiple specialties. Our study retrospectively evaluates the eligibility of patients for MRSA-PreASD in a setting of active targeted MRSA surveillance. We carried out a survey among eligible patients to assess acceptance and feasibility of MRSA-PreASD. Of 10,496 admissions to our university hospital 8912 (84.9%) were screened for MRSA-risk factors. In 5382 admissions at risk swabs were taken and analyzed. Using the Appropriateness Evaluation Protocol (AEP) we retrospectively assessed how many of the 5382 admissions at risk could have been postponed for the duration of an MRSA-PreASD. 36 (17%) of 212 admissions with proven MRSA colonization and 2175 (42%) of 5170 patients without detectable MRSA could have been sent home for MRSA-PreASD to be electively admitted later. Of the 36 admissions (35 patients) with proven MRSA eligible for PreASD 23 patients (65%) responded to an interview. 22 of those (95.6%) would have agreed to PreASD. Additional costs for a screening protocol adapted to the needs of MRSA-PreASD of 52,061€ were estimated. Additional hospitalization costs of 6100-9300€ per MRSA case in Germany have been published. In our study population the successful pre-admission decolonization of 22 cases (63% of 35 patients eligible) may therefore have saved about 134,000-205,000€. Thus from an economic point of view our concept should be justified. In conclusion a relevant number of affected admissions to our tertiary referral hospital is eligible for an MRSA-PreASD. The majority of patients with proven MRSA-colonization eligible for a pre-admission decolonization treatment would prefer such an approach over being isolated at the hospital. The implementation of an MRSA-PreASD-protocol may reduce costs.