RESUMO
BACKGROUND: Frailty is an important predictor of health outcomes, characterized by increased vulnerability due to physiological decline. The Clinical Frailty Scale (CFS) is commonly used for frailty assessment but may be influenced by rater bias. Use of artificial intelligence (AI), particularly Large Language Models (LLMs) offers a promising method for efficient and reliable frailty scoring. METHODS: The study utilized seven standardized patient scenarios to evaluate the consistency and reliability of CFS scoring by OpenAI's GPT-3.5-turbo model. Two methods were tested: a basic prompt and an instruction-tuned prompt incorporating CFS definition, a directive for accurate responses, and temperature control. The outputs were compared using the Mann-Whitney U test and Fleiss' Kappa for inter-rater reliability. The outputs were compared with historic human scores of the same scenarios. RESULTS: The LLM's median scores were similar to human raters, with differences of no more than one point. Significant differences in score distributions were observed between the basic and instruction-tuned prompts in five out of seven scenarios. The instruction-tuned prompt showed high inter-rater reliability (Fleiss' Kappa of 0.887) and produced consistent responses in all scenarios. Difficulty in scoring was noted in scenarios with less explicit information on activities of daily living (ADLs). CONCLUSIONS: This study demonstrates the potential of LLMs in consistently scoring clinical frailty with high reliability. It demonstrates that prompt engineering via instruction-tuning can be a simple but effective approach for optimizing LLMs in healthcare applications. The LLM may overestimate frailty scores when less information about ADLs is provided, possibly as it is less subject to implicit assumptions and extrapolation than humans. Future research could explore the integration of LLMs in clinical research and frailty-related outcome prediction.
RESUMO
BACKGROUND: Discharge letters are a critical component in the continuity of care between specialists and primary care providers. However, these letters are time-consuming to write, underprioritized in comparison to direct clinical care, and are often tasked to junior doctors. Prior studies assessing the quality of discharge summaries written for inpatient hospital admissions show inadequacies in many domains. Large language models such as GPT have the ability to summarize large volumes of unstructured free text such as electronic medical records and have the potential to automate such tasks, providing time savings and consistency in quality. OBJECTIVE: The aim of this study was to assess the performance of GPT-4 in generating discharge letters written from urology specialist outpatient clinics to primary care providers and to compare their quality against letters written by junior clinicians. METHODS: Fictional electronic records were written by physicians simulating 5 common urology outpatient cases with long-term follow-up. Records comprised simulated consultation notes, referral letters and replies, and relevant discharge summaries from inpatient admissions. GPT-4 was tasked to write discharge letters for these cases with a specified target audience of primary care providers who would be continuing the patient's care. Prompts were written for safety, content, and style. Concurrently, junior clinicians were provided with the same case records and instructional prompts. GPT-4 output was assessed for instances of hallucination. A blinded panel of primary care physicians then evaluated the letters using a standardized questionnaire tool. RESULTS: GPT-4 outperformed human counterparts in information provision (mean 4.32, SD 0.95 vs 3.70, SD 1.27; P=.03) and had no instances of hallucination. There were no statistically significant differences in the mean clarity (4.16, SD 0.95 vs 3.68, SD 1.24; P=.12), collegiality (4.36, SD 1.00 vs 3.84, SD 1.22; P=.05), conciseness (3.60, SD 1.12 vs 3.64, SD 1.27; P=.71), follow-up recommendations (4.16, SD 1.03 vs 3.72, SD 1.13; P=.08), and overall satisfaction (3.96, SD 1.14 vs 3.62, SD 1.34; P=.36) between the letters generated by GPT-4 and humans, respectively. CONCLUSIONS: Discharge letters written by GPT-4 had equivalent quality to those written by junior clinicians, without any hallucinations. This study provides a proof of concept that large language models can be useful and safe tools in clinical documentation.
Assuntos
Alta do Paciente , Humanos , Alta do Paciente/normas , Registros Eletrônicos de Saúde/normas , Método Simples-Cego , IdiomaRESUMO
Background: Socioeconomic status (SES) is a well-established determinant of cardiovascular health. However, the relationship between SES and clinical outcomes in long-term out-of-hospital cardiac arrest (OHCA) is less well-understood. The Singapore Housing Index (SHI) is a validated building-level SES indicator. We investigated whether SES as measured by SHI is associated with long-term OHCA survival in Singapore. Methods: We conducted an open cohort study with linked data from the Singapore Pan-Asian Resuscitation Outcomes Study (PAROS), and the Singapore Registry of Births and Deaths (SRBD) from 2010 to 2020. We fitted generalized structural equation models, calculating hazard ratios (HRs) using a Weibull model. We constructed Kaplan-Meier survival curves and calculated the predicted marginal probability for each SHI category. Results: We included 659 cases. In both univariable and multivariable analyses, SHI did not have a significant association with survival. Indirect pathways of SHI mediated through covariates such as Emergency Medical Services (EMS) response time (HR of low-medium, high-medium and high SHI when compared to low SHI: 0.98 (0.88-1.10), 1.01 (0.93-1.11), 1.02 (0.93-1.12) respectively), and age of arrest (HR of low-medium, high-medium and high SHI when compared to low SHI: 1.02 (0.75-1.38), 1.08 (0.84-1.38), 1.18 (0.91-1.54) respectively) had no significant association with OHCA survival. There was no clear trend in the predicted marginal probability of survival among the different SHI categories. Conclusions: We did not find a significant association between SES and OHCA survival outcomes in residential areas in Singapore. Among other reasons, this could be due to affordable healthcare across different socioeconomic classes.
RESUMO
BACKGROUND AND AIM: Colonoscopy is commonly used in screening and surveillance for colorectal cancer. Multiple different guidelines provide recommendations on the interval between colonoscopies. This can be challenging for non-specialist healthcare providers to navigate. Large language models like ChatGPT are a potential tool for parsing patient histories and providing advice. However, the standard GPT model is not designed for medical use and can hallucinate. One way to overcome these challenges is to provide contextual information with medical guidelines to help the model respond accurately to queries. Our study compares the standard GPT4 against a contextualized model provided with relevant screening guidelines. We evaluated whether the models could provide correct advice for screening and surveillance intervals for colonoscopy. METHODS: Relevant guidelines pertaining to colorectal cancer screening and surveillance were formulated into a knowledge base for GPT. We tested 62 example case scenarios (three times each) on standard GPT4 and on a contextualized model with the knowledge base. RESULTS: The contextualized GPT4 model outperformed the standard GPT4 in all domains. No high-risk features were missed, and only two cases had hallucination of additional high-risk features. A correct interval to colonoscopy was provided in the majority of cases. Guidelines were appropriately cited in almost all cases. CONCLUSIONS: A contextualized GPT4 model could identify high-risk features and quote appropriate guidelines without significant hallucination. It gave a correct interval to the next colonoscopy in the majority of cases. This provides proof of concept that ChatGPT with appropriate refinement can serve as an accurate physician assistant.
Assuntos
Colonoscopia , Neoplasias Colorretais , Humanos , Neoplasias Colorretais/diagnóstico , Neoplasias Colorretais/prevenção & controle , Neoplasias Colorretais/epidemiologia , Fatores de Risco , Detecção Precoce de Câncer , AlucinaçõesRESUMO
BACKGROUND: To enhance perioperative outcomes, a perioperative registry that integrates high-quality real-world data throughout the perioperative period is essential. Singapore General Hospital established the Perioperative and Anesthesia Subject Area Registry (PASAR) to unify data from the preoperative, intraoperative, and postoperative stages. This study presents the methodology employed to create this database. METHODS: Since 2016, data from surgical patients have been collected from the hospital electronic medical record systems, de-identified, and stored securely in compliance with privacy and data protection laws. As a representative sample, data from initiation in 2016 to December 2022 were collected. RESULTS: As of December 2022, PASAR data comprise 26 tables, encompassing 153,312 patient admissions and 168,977 operation sessions. For this period, the median age of the patients was 60.0 years, sex distribution was balanced, and the majority were Chinese. Hypertension and cardiovascular comorbidities were also prevalent. Information including operation type and time, intensive care unit (ICU) length of stay, and 30-day and 1-year mortality rates were collected. Emergency surgeries resulted in longer ICU stays, but shorter operation times than elective surgeries. CONCLUSIONS: The PASAR provides a comprehensive and automated approach to gathering high-quality perioperative patient data.
Assuntos
Anestesia , Data Warehousing , Humanos , Pessoa de Meia-Idade , Procedimentos Cirúrgicos Eletivos , Admissão do Paciente , Sistema de RegistrosRESUMO
COVID-19 patients with acute hypoxemic respiratory failure (AHRF) benefit from high flow nasal cannula (HFNC) oxygen therapy. However, delays in initiating invasive ventilation after HFNC failure are associated with poorer outcomes. The respiratory oxygenation (ROX) index, combining SpO2/FiO2 and respiratory rate, can predict HFNC failure. This meta-analysis evaluated the optimal ROX index cut-offs in predicting HFNC failure among COVID-19 patients at different measurement timings and clinical settings. Three databases were searched for eligible papers. From each study, we reconstructed the confusion matrices at different cut-offs, fitted linear mixed models to estimate the ROX index distribution function, and derived the area under the summary receiver operator characteristic curve (sAUC) and optimal cut-offs to predict HFNC failure. 24 studies containing 4790 patients were included. Overall sAUC was 0.771 (95% CI: 0.666-0.847) (optimal cut-off: 5.23, sensitivity: 0.732, specificity: 0.690). The cut-off values to achieve 80%, 90% sensitivity, 80%, 90% specificity were 5.70, 6.69, 4.45, 3.37, respectively. We stratified the analysis by ROX measurement time and estimated optimal cut-offs and cut-offs to achieve 80% sensitivity and specificity. For 2-6 h and 6-12 h post-HFNC initiation, we propose the use of 80% specific cut-offs to rule in HFNC failure of < 5.33 and < 3.69, respectively. For 12-24 h post-HFNC initiation, we propose the use of the 80% sensitive cut-off of > 6.07 to rule out HFNC failure. Our analysis confirms the overall utility of the ROX index in risk stratification of COVID-19 patients with AHRF receiving HFNC and provides potentially useful cut-offs for different times from HFNC initiation.
Assuntos
COVID-19 , Taxa Respiratória , Humanos , Cânula , COVID-19/terapia , Respiração , GasometriaRESUMO
BACKGROUND: Risk stratification in paracetamol (acetaminophen) poisoning is crucial because hepatotoxicity is common and can be mitigated with treatment. However, current risk stratification tools have limitations. AIMS: We evaluated the diagnostic performance of the paracetamol concentration × aminotransferase multiplication product, for predicting hepatotoxicity after paracetamol overdose. METHODS: Medline, Cochrane Library and Embase were searched for eligible papers. We used random effects models to obtain pooled estimates of the likelihood ratios and diagnostic odds ratios, from which sensitivity and specificity were computed. We assessed two commonly used cut-off values of paracetamol × aminotransferase, 1500 mg/L × IU/L and 10,000 mg/L × IU/L. Using the confusion matrices of these two cut-offs, area under the summary receiver operator characteristic curve and optimal cut-off values in different clinical scenarios were established. RESULTS: Six studies comprising 5036 participants were included. In 4051 patients, using the cut-off of 1500 mg/L × IU/L, a diagnostic odds ratio of 31.90 (95%CI: 9.52-106.90), sensitivity of 0.98 (95%CI: 0.94-1.00) and specificity of 0.66 (95%CI: 0.49-0.89) were obtained. In 3983 patients, using the cut-off of 10,000 mg/L × IU/L, a diagnostic odds ratio of 99.34 (95%CI: 12.26-804.87), sensitivity of 0.65 (95%CI: 0.51-0.82) and specificity of 0.97 (95%CI: 0.95-1.00) were obtained. For staggered ingestions, the 1500 mg/L × IU/L cut-off yielded a diagnostic odds ratio of 69.53 (95%CI: 4.03-1199.75), sensitivity of 1.00 (95%CI: 0.87-1.00) and specificity of 0.74 (95%CI: 0.43-1.00). Next, using the 10,000 mg/L × IU/L cut-off in this scenario yielded a diagnostic odds ratio of 254.58 (95%CI: 11.12-5827.60), sensitivity of 0.79 (95%CI: 0.59-1.00) and specificity of 0.98 (95%CI: 0.94-1.00). The overall summary receiver operator characteristic curve was 0.91 (95%CI: 0.75-0.97), and the optimal cut-off value was 3840 mg/L × IU/L. The summary receiver operator characteristic curve in patients with staggered ingestions was 0.96 (95%CI: 0.85-0.99). The summary receiver operator characteristic curve in patients with staggered ingestions and whose paracetamol concentration was below the detectable limit of 10 mg/L at presentation was 0.97 (95%CI: 0.94-0.99). CONCLUSION: In this first meta-analysis, paracetamol × aminotransferase demonstrates its use in prognosticating hepatotoxicity in patients with paracetamol poisoning. It complements the Rumack-Matthew nomogram as it has shown promise in addressing two key limitations of the nomogram: it is usable after more than 24 h between overdose and acetylcysteine treatment, and it is applicable in staggered ingestions.
Assuntos
Analgésicos não Narcóticos , Doença Hepática Induzida por Substâncias e Drogas , Overdose de Drogas , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Humanos , Acetaminofen , Doença Hepática Induzida por Substâncias e Drogas/diagnóstico , Doença Hepática Induzida por Substâncias e Drogas/etiologia , Doença Hepática Induzida por Substâncias e Drogas/tratamento farmacológico , Alanina Transaminase , Overdose de Drogas/diagnóstico , Overdose de Drogas/tratamento farmacológico , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos/tratamento farmacológico , Medição de Risco , Estudos RetrospectivosRESUMO
OBJECTIVE: Understanding the social determinants of bystander cardiopulmonary resuscitation (CPR) receipt can inform the design of public health interventions to increase bystander CPR. The association of socioeconomic status with bystander CPR is generally poorly understood. We evaluated the relationship between socioeconomic status and bystander CPR in cases of out-of-hospital cardiac arrest (OHCA). METHODS: This was a retrospective cohort study based on the Singapore cohort of the Pan-Asian Resuscitation Outcomes Study registry between 2010 and 2018. We categorized patients into low, medium, and high Singapore Housing Index (SHI) levels-a building-level index of socioeconomic status. The primary outcome was receipt of bystander CPR. The secondary outcomes were prehospital return of spontaneous circulation and survival to discharge. RESULTS: A total of 12,730 OHCA cases were included, the median age was 71 years, and 58.9% were male. The bystander CPR rate was 56.7%. Compared to patients in the low SHI category, those in the medium and high SHI categories were more likely to receive bystander CPR (medium SHI: adjusted odds ratio [aOR] 1.48, 95% CI 1.30-1.69; high SHI: aOR 1.93, 95% CI 1.67-2.24). High SHI patients had higher survival compared to low SHI patients on unadjusted analysis (OR 1.79, 95% CI 1.08-2.96), but not adjusted analysis (adjusted for age, sex, race, witness status, arrest time, past medical history of cancer, and first arrest rhythm). When comparing high with low SHI, females had larger increases in bystander CPR rates than males. CONCLUSIONS: Lower building-level socioeconomic status was independently associated with lower rate of bystander CPR, and females were more susceptible to the effect of low socioeconomic status on lower rate of bystander CPR.
Assuntos
Reanimação Cardiopulmonar , Serviços Médicos de Emergência , Parada Cardíaca Extra-Hospitalar , Feminino , Humanos , Masculino , Idoso , Estudos Retrospectivos , Coleta de Dados , Classe Social , Parada Cardíaca Extra-Hospitalar/terapiaRESUMO
The American Society of Anesthesiologists Physical Status Classification (ASA) is used for communication of patient health status, risk scoring, benchmarking and financial claims. Prior studies using hypothetical scenarios have shown poor concordance of ASA classification among healthcare providers. There is a paucity of studies using clinical data, and of clinical factors or patient outcomes associated with discordant classification. The study aims to assess ASA classification concordance between surgeons and anesthesiologists, factors surrounding discordance and its impact on patient outcomes. This retrospective cohort study was conducted in a tertiary medical center on 46,284 consecutive patients undergoing elective surgery between January 2017 and December 2019. The ASA class showed moderate concordance (weighted Cohen's κ 0.53) between surgeons and anesthesiologists. We found significant associations between discordant classification and patient comorbidities, age and race. Patients with discordant classification had a higher risk of 30-day mortality (odds ratio (OR) 2.00, 95% confidence interval (CI) = 1.52-2.62, p < 0.0001), 1-year mortality (OR 1.53, 95% CI = 1.38-1.69, p < 0.0001), and Intensive Care Unit admission > 24 h (OR 1.69, 95% CI = 1.47-1.94, p < 0.0001). Hence, there is a need for improved standardization of ASA scoring and cross-specialty review in ASA-discordant cases.
Assuntos
Anestesiologistas , Cirurgiões , Procedimentos Cirúrgicos Eletivos/efeitos adversos , Humanos , Razão de Chances , Estudos Retrospectivos , Estados Unidos/epidemiologiaRESUMO
BACKGROUND: Socioeconomic status (SES) is an important determinant of health, and SES data is an important confounder to control for in epidemiology and health services research. Individual level SES measures are cumbersome to collect and susceptible to biases, while area level SES measures may have insufficient granularity. The 'Singapore Housing Index' (SHI) is a validated, building level SES measure that bridges individual and area level measures. However, determination of the SHI has previously required periodic data purchase and manual parsing. In this study, we describe a means of SHI determination for public housing buildings with open government data, and validate this against the previous SHI determination method. METHODS: Government open data sources (e.g. DATA: gov.sg, Singapore Land Authority OneMAP API, Urban Redevelopment Authority API) were queried using custom Python scripts. Data on residential public housing block address and composition from the HDB Property Information dataset (data.gov.sg) was matched to postal code and geographical coordinates via OneMAP API calls. The SHI was calculated from open data, and compared to the original SHI dataset that was curated from non-open data sources in 2018. RESULTS: Ten thousand seventy-seven unique residential buildings were identified from open data. OneMAP API calls generated valid geographical coordinates for all (100%) buildings, and valid postal code for 10,012 (99.36%) buildings. There was an overlap of 10,011 buildings between the open dataset and the original SHI dataset. Intraclass correlation coefficient was 0.999 for the two sources of SHI, indicating almost perfect agreement. A Bland-Altman plot analysis identified a small number of outliers, and this revealed 5 properties that had an incorrect SHI assigned by the original dataset. Information on recently transacted property prices was also obtained for 8599 (85.3%) of buildings. CONCLUSION: SHI, a useful tool for health services research, can be accurately reconstructed using open datasets at no cost. This method is a convenient means for future researchers to obtain updated building-level markers of socioeconomic status for policy and research.
Assuntos
Habitação , Classe Social , Pesquisa sobre Serviços de Saúde , Humanos , SingapuraRESUMO
PURPOSE: To describe the inception and structure of the SingHealth Diabetes Registry (SDR) as well as the methodology used to set up the registry. The SDR was established to facilitate systematic and standardized data collection for diabetes mellitus within Singapore Health Services (SingHealth), which is an Academic Medical Center (AMC) and Singapore's largest group of healthcare institutions. The diabetes casemix and outcome variables within the registry cohort are also provided. MATERIALS AND METHODS: The SDR is built from SingHealth's electronic medical records (EMR) and clinical databases. It covers all individuals aged 18 and above with diabetes mellitus, excluding those with pre-diabetes. Cases are annually ascertained using criteria that include diagnosis codes, prescription records and laboratory test records. Data collection of casemix and outcome variables for the period 2013 to 2019 is complete. RESULTS: The SDR stands at 208,102 ascertained individuals, distributed across 8 healthcare sites within the AMC. The cohort is broadly reflective of the local gender and ethnic compositions but has a high proportion of older individuals with a mean age of 65.8 ± 13.7 years. Majority (>99%) have type 2 diabetes mellitus, with multiple other comorbidities (hypertension 84.1%, hyperlipidemia 86.2%, established cardiovascular disease 34.1%). At present, majority of individuals are able to meet key process indicators and 52.7% have a mean HbA1c of <7% (53 mmol/mol). Areas of potential improvement include increasing eye and foot screening rates, as well as glycemic control for the 19.5% of individuals with mean HbA1c >8% (64 mmol/mol). CONCLUSION: The SDR is a large-scale, comprehensive, and representative diabetes registry that incorporates EMR data across the primary and hospital-based care continuum, in a major AMC in Singapore. The SDR has identified areas of improvement in diabetes processes and outcomes. It will support future quality assessment and improvements in diabetes care.
RESUMO
BACKGROUND: Palliative gastrointestinal (GI) surgery potentially relieves distressing symptoms arising from intestinal obstruction (IO) in patients with advanced peritoneal carcinomatosis (PC). As surgery is associated with significant morbidity risks in advanced cancer patients, it is important for surgeons to select patients who can benefit the most from this approach. Hence, we aim to determine predictors of morbidity and mortality after palliative surgery in patients with PC. In addition, we evaluate the utility of the UC Davis Cancer Care nomogram (UCDCCn) and develop a simplified model to predict short-term surgical mortality in these patients. METHODS: A retrospective review of patients with IO secondary to PC undergoing palliative GI surgery was performed. Logistic regression was used to determine independent predictors of 30-day morbidity and mortality after surgery. UCDCCn was evaluated using the area under the curve (AUC) for discriminatory power and the Hosmer-Lemeshow test for calibration. Our simplified model was developed using logistic regression and evaluated using cross-validation. RESULTS: A total of 254 palliative GI surgeries were performed over a 10-year duration. The 30-day morbidity and mortality were 43% (n = 110) and 21% (n = 53), respectively. Preoperative albumin, age, and emergency nature of surgery were significant independent predictors for 30-day morbidity. A simplified model using preoperative Eastern Cooperative Oncology Group (ECOG) status and albumin (AUC = 0.71) achieved better predictive power than UCDCCn (AUC = 0.66) for 30-day mortality. CONCLUSION: Good ECOG status and high preoperative albumin levels were independently associated with good short-term outcomes after palliative GI surgery. Our simplified model may be used to conveniently and efficiently select patients who stand to benefit the most from surgery.