RESUMEN
Importance: Although augmenting large language models (LLMs) with knowledge bases may improve medical domain-specific performance, practical methods are needed for local implementation of LLMs that address privacy concerns and enhance accessibility for health care professionals. Objective: To develop an accurate, cost-effective local implementation of an LLM to mitigate privacy concerns and support their practical deployment in health care settings. Design, Setting, and Participants: ChatZOC (Sun Yat-Sen University Zhongshan Ophthalmology Center), a retrieval-augmented LLM framework, was developed by enhancing a baseline LLM with a comprehensive ophthalmic dataset and evaluation framework (CODE), which includes over 30â¯000 pieces of ophthalmic knowledge. This LLM was benchmarked against 10 representative LLMs, including GPT-4 and GPT-3.5 Turbo (OpenAI), across 300 clinical questions in ophthalmology. The evaluation, involving a panel of medical experts and biomedical researchers, focused on accuracy, utility, and safety. A double-masked approach was used to try to minimize bias assessment across all models. The study used a comprehensive knowledge base derived from ophthalmic clinical practice, without directly involving clinical patients. Exposures: LLM response to clinical questions. Main Outcomes and Measures: Accuracy, utility, and safety of LLMs in responding to clinical questions. Results: The baseline model achieved a human ranking score of 0.48. The retrieval-augmented LLM had a score of 0.60, a difference of 0.12 (95% CI, 0.02-0.22; P = .02) from baseline and not different from GPT-4 with a score of 0.61 (difference = 0.01; 95% CI, -0.11 to 0.13; P = .89). For scientific consensus, the retrieval-augmented LLM was 84.0% compared with the baseline model of 46.5% (difference = 37.5%; 95% CI, 29.0%-46.0%; P < .001) and not different from GPT-4 with a value of 79.2% (difference = 4.8%; 95% CI, -0.3% to 10.0%; P = .06). Conclusions and Relevance: Results of this quality improvement study suggest that the integration of high-quality knowledge bases improved the LLM's performance in medical domains. This study highlights the transformative potential of augmented LLMs in clinical practice by providing reliable, safe, and practical clinical information. Further research is needed to explore the broader application of such frameworks in the real world.
RESUMEN
BACKGROUND/AIMS: The aim of this study was to develop and evaluate digital ray, based on preoperative and postoperative image pairs using style transfer generative adversarial networks (GANs), to enhance cataractous fundus images for improved retinopathy detection. METHODS: For eligible cataract patients, preoperative and postoperative colour fundus photographs (CFP) and ultra-wide field (UWF) images were captured. Then, both the original CycleGAN and a modified CycleGAN (C2ycleGAN) framework were adopted for image generation and quantitatively compared using Frechet Inception Distance (FID) and Kernel Inception Distance (KID). Additionally, CFP and UWF images from another cataract cohort were used to test model performances. Different panels of ophthalmologists evaluated the quality, authenticity and diagnostic efficacy of the generated images. RESULTS: A total of 959 CFP and 1009 UWF image pairs were included in model development. FID and KID indicated that images generated by C2ycleGAN presented significantly improved quality. Based on ophthalmologists' average ratings, the percentages of inadequate-quality images decreased from 32% to 18.8% for CFP, and from 18.7% to 14.7% for UWF. Only 24.8% and 13.8% of generated CFP and UWF images could be recognised as synthetic. The accuracy of retinopathy detection significantly increased from 78% to 91% for CFP and from 91% to 93% for UWF. For retinopathy subtype diagnosis, the accuracies also increased from 87%-94% to 91%-100% for CFP and from 87%-95% to 93%-97% for UWF. CONCLUSION: Digital ray could generate realistic postoperative CFP and UWF images with enhanced quality and accuracy for overall detection and subtype diagnosis of retinopathies, especially for CFP.\ TRIAL REGISTRATION NUMBER: This study was registered with ClinicalTrials.gov (NCT05491798).
RESUMEN
AIMS: To explore the possibility of implementing Choosing Wisely on ocular patients in China by investigating the prevalence of abnormalities in routine preoperative blood tests (RPBTs) and its turnaround time (TAT). METHODS: Data from 102 542 ocular patients between January 2016 and December 2018, at Zhongshan Ophthalmic Center, were pooled from the laboratory information system. The test results were divided into normal and abnormal, including critical values. Ocular diseases were stratified into 11 subtypes based on the primary diagnosis. The TAT of 243 350 blood tests from January 2017 to December 2018 was categorised into transportation time and intralaboratory time. RESULTS: RPBT was grouped into complete blood count (CBC), blood biochemistry (BBC), blood coagulation (BCG) and blood-borne pathogens (BBP), completed for 97.22%, 87.66%, 94.41% and 95.35% of the recruited patients (male, 52 549 (51.25%); median(IQR) age, 54 (29-67) years), respectively. Stratified by the test items, 9.19% (95% CI 9.07% to 9.31%) were abnormal results, and 0.020% (95% CI 0.019% to 0.022%) were critical; most abnormalities were on the CBC, while glucose was the most common critical item. Classified by the patients' primary diagnosis, 76.97% (95% CI 76.71% to 77.23%) had at least one abnormal result, and 0.28% (95% CI 0.25% to 0.32%) were critical; abnormal findings were reported in 45.29% (95% CI 44.98% to 45.60%), 54.97% (95% CI 54.65% to 55.30%), 30.29% (95% CI 30.00% to 30.58%) and 11.32% (95% CI 11.12% to 11.52%) for the CBC, BBC, BCG and BBP tests, respectively. The median transportation time and intralaboratory TAT of the samples were 12 min and 78 min respectively. CONCLUSION: Blood abnormalities are common in ocular patients. With acceptable timelines, RPBT is still indispensable in China for patient safety.
