Leveraging Large Language Models for Precision Monitoring of Chemotherapy-Induced Toxicities: A Pilot Study with Expert Comparisons and Future Directions.

INTRODUCTION: Large Language Models (LLMs), such as the GPT model family from OpenAI, have demonstrated transformative potential across various fields, especially in medicine. These models can understand and generate contextual text, adapting to new tasks without specific training. This versatility can revolutionize clinical practices by enhancing documentation, patient interaction, and decision-making processes. In oncology, LLMs offer the potential to significantly improve patient care through the continuous monitoring of chemotherapy-induced toxicities, which is a task that is often unmanageable for human resources alone. However, existing research has not sufficiently explored the accuracy of LLMs in identifying and assessing subjective toxicities based on patient descriptions. This study aims to fill this gap by evaluating the ability of LLMs to accurately classify these toxicities, facilitating personalized and continuous patient care. METHODS: This comparative pilot study assessed the ability of an LLM to classify subjective toxicities from chemotherapy. Thirteen oncologists evaluated 30 fictitious cases created using expert knowledge and OpenAI's GPT-4. These evaluations, based on the CTCAE v.5 criteria, were compared to those of a contextualized LLM model. Metrics such as mode and mean of responses were used to gauge consensus. The accuracy of the LLM was analyzed in both general and specific toxicity categories, considering types of errors and false alarms. The study's results are intended to justify further research involving real patients. RESULTS: The study revealed significant variability in oncologists' evaluations due to the lack of interaction with fictitious patients. The LLM model achieved an accuracy of 85.7% in general categories and 64.6% in specific categories using mean evaluations with mild errors at 96.4% and severe errors at 3.6%. False alarms occurred in 3% of cases. When comparing the LLM's performance to that of expert oncologists, individual accuracy ranged from 66.7% to 89.2% for general categories and 57.0% to 76.0% for specific categories. The 95% confidence intervals for the median accuracy of oncologists were 81.9% to 86.9% for general categories and 67.6% to 75.6% for specific categories. These benchmarks highlight the LLM's potential to achieve expert-level performance in classifying chemotherapy-induced toxicities. DISCUSSION: The findings demonstrate that LLMs can classify subjective toxicities from chemotherapy with accuracy comparable to expert oncologists. The LLM achieved 85.7% accuracy in general categories and 64.6% in specific categories. While the model's general category performance falls within expert ranges, specific category accuracy requires improvement. The study's limitations include the use of fictitious cases, lack of patient interaction, and reliance on audio transcriptions. Nevertheless, LLMs show significant potential for enhancing patient monitoring and reducing oncologists' workload. Future research should focus on the specific training of LLMs for medical tasks, conducting studies with real patients, implementing interactive evaluations, expanding sample sizes, and ensuring robustness and generalization in diverse clinical settings. CONCLUSIONS: This study concludes that LLMs can classify subjective toxicities from chemotherapy with accuracy comparable to expert oncologists. The LLM's performance in general toxicity categories is within the expert range, but there is room for improvement in specific categories. LLMs have the potential to enhance patient monitoring, enable early interventions, and reduce severe complications, improving care quality and efficiency. Future research should involve specific training of LLMs, validation with real patients, and the incorporation of interactive capabilities for real-time patient interactions. Ethical considerations, including data accuracy, transparency, and privacy, are crucial for the safe integration of LLMs into clinical practice.

Predicting Severe Haematological Toxicity in Gastrointestinal Cancer Patients Undergoing 5-FU-Based Chemotherapy: A Bayesian Network Approach.

PURPOSE: Severe toxicity is reported in about 30% of gastrointestinal cancer patients receiving 5-Fluorouracil (5-FU)-based chemotherapy. To date, limited tools exist to identify at risk patients in this setting. The objective of this study was to address this need by designing a predictive model using a Bayesian network, a probabilistic graphical model offering robust, explainable predictions. METHODS: We utilized a dataset of 267 gastrointestinal cancer patients, conducting preprocessing, and splitting it into TRAIN and TEST sets (80%:20% ratio). The RandomForest algorithm assessed variable importance based on MeanDecreaseGini coefficient. The bnlearn R library helped design a Bayesian network model using a 10-fold cross-validation on the TRAIN set and the aic-cg method for network structure optimization. The model's performance was gauged based on accuracy, sensitivity, and specificity, using cross-validation on the TRAIN set and independent validation on the TEST set. RESULTS: The model demonstrated satisfactory performance with an average accuracy of 0.85 (±0.05) and 0.80 on TRAIN and TEST datasets, respectively. The sensitivity and specificity were 0.82 (±0.14) and 0.87 (±0.07) for the TRAIN dataset, and 0.71 and 0.83 for the TEST dataset, respectively. A user-friendly tool was developed for clinical implementation. CONCLUSIONS: Despite several limitations, our Bayesian network model demonstrated a high level of accuracy in predicting the risk of developing severe haematological toxicity in gastrointestinal cancer patients receiving 5-FU-based chemotherapy. Future research should aim at model validation in larger cohorts of patients and different clinical settings.

A Screening Tool for Self-Evaluation of Risk for Age-Related Macular Degeneration: Validation in a Spanish Population.

Purpose: The objectives of this study were the creation and validation of a screening tool for age-related macular degeneration (AMD) for routine assessment by primary care physicians, ophthalmologists, other healthcare professionals, and the general population. Methods: A simple, self-administered questionnaire (Simplified Théa AMD Risk-Assessment Scale [STARS] version 4.0) which included well-established risk factors for AMD, such as family history, smoking, and dietary factors, was administered to patients during ophthalmology visits. A fundus examination was performed to determine presence of large soft drusen, pigmentary abnormalities, or late AMD. Based on data from the questionnaire and the clinical examination, predictive models were developed to estimate probability of the Age-Related Eye Disease Study (AREDS) score (categorized as low risk/high risk). The models were evaluated by area under the receiving operating characteristic curve analysis. Results: A total of 3854 subjects completed the questionnaire and underwent a fundus examination. Early/intermediate and late AMD were detected in 15.9% and 23.8% of the patients, respectively. A predictive model was developed with training, validation, and test datasets. The model in the test set had an area under the curve of 0.745 (95% confidence interval [CI] = 0.705-0.784), a positive predictive value of 0.500 (95% CI = 0.449-0.557), and a negative predictive value of 0.810 (95% CI = 0.770-0.844). Conclusions: The STARS questionnaire version 4.0 and the model identify patients at high risk of developing late AMD. Translational Relevance: The screening instrument described could be useful to evaluate the risk of late AMD in patients >55 years without having an eye examination, which could lead to more timely referrals and encourage lifestyle changes.