Comparison of the Quality of Discharge Letters Written by Large Language Models and Junior Clinicians: Single-Blinded Study.

BACKGROUND: Discharge letters are a critical component in the continuity of care between specialists and primary care providers. However, these letters are time-consuming to write, underprioritized in comparison to direct clinical care, and are often tasked to junior doctors. Prior studies assessing the quality of discharge summaries written for inpatient hospital admissions show inadequacies in many domains. Large language models such as GPT have the ability to summarize large volumes of unstructured free text such as electronic medical records and have the potential to automate such tasks, providing time savings and consistency in quality. OBJECTIVE: The aim of this study was to assess the performance of GPT-4 in generating discharge letters written from urology specialist outpatient clinics to primary care providers and to compare their quality against letters written by junior clinicians. METHODS: Fictional electronic records were written by physicians simulating 5 common urology outpatient cases with long-term follow-up. Records comprised simulated consultation notes, referral letters and replies, and relevant discharge summaries from inpatient admissions. GPT-4 was tasked to write discharge letters for these cases with a specified target audience of primary care providers who would be continuing the patient's care. Prompts were written for safety, content, and style. Concurrently, junior clinicians were provided with the same case records and instructional prompts. GPT-4 output was assessed for instances of hallucination. A blinded panel of primary care physicians then evaluated the letters using a standardized questionnaire tool. RESULTS: GPT-4 outperformed human counterparts in information provision (mean 4.32, SD 0.95 vs 3.70, SD 1.27; P=.03) and had no instances of hallucination. There were no statistically significant differences in the mean clarity (4.16, SD 0.95 vs 3.68, SD 1.24; P=.12), collegiality (4.36, SD 1.00 vs 3.84, SD 1.22; P=.05), conciseness (3.60, SD 1.12 vs 3.64, SD 1.27; P=.71), follow-up recommendations (4.16, SD 1.03 vs 3.72, SD 1.13; P=.08), and overall satisfaction (3.96, SD 1.14 vs 3.62, SD 1.34; P=.36) between the letters generated by GPT-4 and humans, respectively. CONCLUSIONS: Discharge letters written by GPT-4 had equivalent quality to those written by junior clinicians, without any hallucinations. This study provides a proof of concept that large language models can be useful and safe tools in clinical documentation.

Hybrid architecture based intelligent diagnosis assistant for GP.

As the first point of contact for patients, General Practitioners (GPs) play a crucial role in the National Health Service (NHS). An accurate primary diagnosis from the GP can alleviate the burden on specialists and reduce the time needed to re-confirm the patient's condition, allowing for more efficient further examinations. However, GPs have broad but less specialized knowledge, which limits the accuracy of their diagnosis. Therefore, it is imperative to introduce an intelligent system to assist GPs in making decisions. This paper introduces two data augmentation methods, the Complaint Symptoms Integration Method and Symptom Dot Separating Method, to integrate essential information into the Integration dataset. Additionally, it proposes a hybrid architecture that fuses the features of words from different representation spaces. Experiments demonstrate that, compared to commonly used pre-trained attention-based models, our hybrid architecture delivers the best classification performance for four common neurological diseases on the enhanced Integration dataset. For example, the classification accuracy of the BERT+CNN hybrid architecture is 0.897, which is a 5.1% improvement over both BERT and CNN with 0.846. Finally, this paper develops an AI diagnosis assistant web application that leverages the superior performance of this architecture to help GPs complete primary diagnosis efficiently and accurately.

Informative value of referral letters from general practice for child and adolescent mental healthcare.

Although referral letters (RLs) form a nodal point in a patient's care journey, little is known about their informative value in child and adolescent mental healthcare. To determine the informative value of RLs to child and adolescent psychiatry, we conducted a chart review in medical records of minors registered at specialized mental healthcare between January 2015 and December 2017 (The Netherlands). Symptoms indicated in RLs originating from general practice (N = 723) were coded and cross-tabulated with the best estimate clinical classifications made in psychiatry. Results revealed that over half of the minors in the sample were classified in concordance with at least one reason for referral. We found fair to excellent discriminative ability for indications made in RLs concerning the most common psychiatric classifications (95% CI AUC: 60.9-70.6 for anxiety disorders to 90.5-100.0 for eating disorders). Logistic regression analyses suggested no statistically significant effects of gender, age, severity or mental healthcare history, with the exception of age and attention deficit hyperactivity disorders (ADHD), as RLs better predicted ADHD with increasing age (OR = 1.14, 95% CI 1.03-1.27). Contextual problems, such as difficulties studying, problems with parents or being bullied were indicated frequently and associated with classifications in various disorder groups. To conclude, general practitioners' RLs showed informative value, contrary to common beliefs. Replication studies are needed to reliably incorporate RLs into the diagnostic work-up.

Quality and appropriateness of uro-surgical referrals at a Nigerian teaching hospital.

Introduction: An effective referral system is crucial for a high-quality health system that provides safe medical care. Aim and Objectives: This study aimed at evaluating the appropriateness and adequacies of information in the referral letters of patients. Materials and Methods: A prospective study of referral letters of all new patients in the urology clinic. Information retrieved was socio-demographic characteristics, source of referral and presence or absence of important information in their letters. We compared the information provided to the new history taken to determine the appropriateness and adequacy using different domains of medical history. Referrals were adjudged appropriate if the diagnosis is urologic, while any referral lacking relevant information is adjudged inadequate. The results were displayed using the simple proportions in tables and charts. Results: A total of 1188 referrals were reviewed. There were 997 (83.9%) males and 191 (16.1%) females. Referrals from private hospitals were the most common in 627 (52.8%) cases. Of all new referrals, 1165 (98.1%) were adjudged to be appropriate, while 23 (1.9%) were inappropriately referred. Referrals from teaching hospitals had higher proportions of good-quality referrals than those from primary healthcare and private centres. The most common deficiencies were the lack of documentation of relevant examination findings (37.8%) and provisional diagnosis (21.4%). The majority, 956 (80.5%), of letters were narrative in nature, whereas 232 (19.5%) were structured. Structured letters were found to be more informative. Conclusion: A significant percentage of referral letters lacked completeness in many key areas. We recommend the use of structured forms or template letters to improve the quality of referrals.

Referrals to a pediatric emergency department of a tertiary care teaching hospital before and after introduction of a referral education module - a quality improvement study.

BACKGROUND: Provision of timely care to critically ill children is essential for good outcome. Referral from smaller peripheral hospitals to higher centers for intensive care is common. However, lack of an organized referral and feedback system compromises optimal care. We studied the quality of referral letters coming to our Emergency Department (ED) with respect to their demography, association with severity of illness and mortality before and after referral education. METHODS: Our study was completed in three phases in the Pediatric ED; Pre-intervention, Intervention and Post intervention phases. Quality of referral letter was matched with a quality checklist proforma and graded as 'good', 'fair' and 'poor' if it scored > 7, 5-7 and < 5 points respectively. A peer reviewed referral education module was prepared using case studies, expert opinions, and lacunae observed in the first phase and administered to health care providers (HCP's) of referring hospitals. Quality of referral letter was compared between pre and post intervention phases. RESULTS: Most referrals belonged to the neighboring states of Punjab (48.2%) and Haryana (22.4%). Major referring hospitals were from public sector (80.9%), of which the teaching hospitals topped the list (53.6%). Government run ambulance services (85.5%) was commonest mode of transport used and need for a PICU bed and/or mechanical ventilation (50.4%) was the commonest reason for referral. The post intervention phase saw a significant decline in the proportion of poor (93.2 vs.78.2%; p = 0.001) and a significant increase in the proportion of fair (6.1 vs 18%; p = 0.001) and good referral letters (0.7 vs 18%; p = 0.001). The proportion of children with physiological decompensation at triage had reduced significantly in the post intervention phase [513 out of 1403 (36.5%) vs. 310 out of 957 (32.3%); p = 0.001]. CONCLUSION: Referral education had significantly improved the quality of referral letters. Proportion of children with physiological decompensation at triage had decreased significantly after referral module. This change suggests sensitization of the peripheral hospitals towards a better referral process. Continued multifaceted approach will be required for sustained and increased benefits.

Utility of a Referral Letter to Improve Comprehensibility of Cancer Patients in Palliative Care: a Single-Center Study.

In spite of a referral letter as an important document for communicating between physicians, whether it could also be useful as a source of information for patients has not yet established. We included cancer patients in palliative care setting, all of whom completed a standardized questionnaire regarding their opinion concerning the utility of a referral letter as a source of information and its requirements to achieve a better understanding. Completed questionnaires were received from 50 cancer patients. Ninety-four percent of participants agreed that a referral letter could be of great importance for procuring medical information to them. There was only minor divergence among the participants respecting age, gender, or education. Particular requirements were diagnosis, treatment plan, prognosis, list of drugs, and contact data of involved physicians. Additional important topics were laboratory values, alternatives to current therapy, side effects and supportive therapy, and advices regarding lifestyle and naturopathy. The majority of patients also concluded to accept technical terms in doctor's letters if a glossary supported their comprehension. The majority of patients prefer a concise description of medical information in a referral letter. This form of a letter would boost patients' involvement and help them transfer medical information to other therapists or relatives.

How many paediatric referrals to an allergist could be managed by a general practitioner with special interest?

BACKGROUND: The concept of a general practitioner with special interest (GPwSI) was first proposed in the 2000 National Health Service Plan, as a way of providing specialized treatment closer to the patient's home and reducing hospital waiting times. Given the patchy and inadequate provision of paediatric allergy services in the UK, the introduction of GPwSIs might reduce pressure on existing specialist services. METHODS: A total of 100 consecutive referrals to a specialist paediatric allergy clinic were reviewed to assess what proportion could be managed by a GPwSI allergy service with a predefined range of facilities and expertise (accurate diagnosis and management of allergy; skin prick testing; provision of allergen avoidance advice; ability to assess suitability for desensitization). Each referral was reviewed independently by three allergy specialists. Cases were initially judged on the referral letter and then, to determine whether appropriate triage decisions could be made prospectively, cases were re-assessed with information summarized in the clinic letter. The proportion of referrals suitable for a GPwSI was calculated and their characteristics identified. RESULTS: At least 42% and up to 75% were suitable for management by a GPwSI in allergy based on unanimous and majority agreement, respectively. The appropriateness of 79% referrals could be identified based on the information in the referral letter. A total of 19% referrals were unsuitable for a GPwSI service because of complex or multisystem disease, need for specialist knowledge or facilities or patient's young age. CONCLUSIONS: At least two-fifths of paediatric allergy referrals to our hospital-based service could be dealt with in a GPwSI clinic, thereby diversifying the patient pathway, allowing specialist services to focus on complex cases and reducing waiting times for appointments.

Is the devil in the detail? The quality and clinical impact of information provided on requests for non-trauma emergency abdominal CT scans.

BACKGROUND: The role of the radiology information system in streamlining imaging workflow and enhancing efficiency in digital radiology departments is now well established. Although there is increasing use of systems with the capacity for computerized physician order entry, there has been limited work on the quality of clinical data provided on electronic diagnostic imaging requests. PURPOSE: To assess the quality of clinical details provided on electronic diagnostic imaging requests (DIR) for emergency non-trauma abdominal computed tomography (CT) scans, and the impact of such data on radiological outcomes. MATERIAL AND METHODS: We conducted a retrospective analysis of 100 consecutive electronic DIRs for emergency non-trauma abdominal CT scans for patients with an acute abdomen in a tertiary-level public-sector hospital. The quality of clinical data was assessed using the Royal College of Physicians' referral guidelines and correlated with radiological outcomes, defined as a definitive CT diagnosis. RESULTS: Eighty-eight percent of requests presented a clear clinical question, 48% recorded clinical examination details, 29% had adequate clinical histories, and 17% included laboratory investigations, while only 2% of requests were complete in all respects. Although 88% of scans yielded a definitive radiological diagnosis, there was no association between the adequacy of DIR details and a definitive radiological outcome. CONCLUSION: Our findings underscore the non-specific clinical presentation of non-trauma-related abdominal emergencies and the pivotal role of CT in providing a definitive diagnosis in this setting. We suggest that the appropriate triage of patients presenting with an acute, non-trauma abdomen is the overriding clinical imperative.

Essential information for transition of care for frail elderly patients in Japan: A qualitative study.

BACKGROUND: Information exchange between hospitals and primary care physicians is suboptimal. Most physicians are dissatisfied with the current referral process, and poor communication leads to negative care transition outcomes. METHOD: To identify the key information needed for a successful transition of care, we conducted a qualitative study using consecutive, semistructured in-person interviews and focus group sessions. We recruited five participants engaged in clinical work for individual interviews and 16 participants for focus groups. We analyzed all data using qualitative thematic analysis. All results were returned to the participants and modified based on their feedback. RESULTS: The five individual interviews provided a general picture of the current referral process and an interview guide for the following focus group sessions. The focus group discussions were used to identify the essential information needed at admission and discharge from the hospital. Essential information on hospital admission was as follows: (1) basic medical and care information, (2) care resources available at home, (3) the purpose of admission and the goals of care during hospitalization, and (4) status of advance care planning (ACP) and patient's will in an emergency. Essential information on hospital discharge was as follows: (1) clinical course, (2) explanation of medical condition during hospitalization, (3) status of ACP and patient's will in an emergency, and (4) medical procedures to be continued at home. CONCLUSIONS: We identified the essential information needed for a successful transition of care in Japan. The clinical effectiveness of a template that contains the information identified in our study warrants further investigation.

Telemedicine or In-Person: Referral Letter Content Influencing How a Patient Receives Treatment.

Objective: This study investigated hospital-based specialist services that provide both traditional hospital outpatient appointments (in-person) or through a live videoconferencing session (telehealth) to referred patients. Referral letters submitted to these clinics were assessed against an inclusion criterion and grouped according to which of delivery method the patient received for their appointment (in-person or telehealth). These groups were then compared for differences to see what factors, if any, influence the likelihood of a patient being offered a telehealth appointment. Methods: An extract of all referral letters meeting inclusion criteria between July 01, 2019 and June 30, 2020 were collected (n = 441). Letters were grouped according to delivery modality (in-person or telehealth) and differences between the groups, including variables such as patient demographics, clinical condition, and urgency and the reviewing clinician were assessed for associations. Results: This study observed that where the referring clinician suggested a telehealth appointment for their patient, this was more likely to be offered (38.25%) compared with referrals that did not (7.36%) (x 2 1 = 28.33, p = 0.1857, odds ratio = 2.77). Patients were more likely to be offered a telehealth appointment the further they lived from the treating facility (T = -4.51 on 106.59 df, p = 1.622 e-05). Variation in the selection of delivery modality among reviewing clinicians was also observed (x 2 1 = 42.334, p < 1.42e-08). Discussion: Existing research indicates there is a strong link between the perceptions clinicians as individuals have of telehealth and a willingness to offer this modality to patients. Despite this, specific information about a patient contained within a referral letter may influence the delivery modality that the patient will be offered for their initial appointment. It is important that this information is more routinely included in letters sent by referring clinicians to hospital-based specialist services. It is equally important that when included, this information is identified and actioned by reviewing clinicians in a consistent way. Doing so will benefit patients by increasing the likelihood that they will receive specialist outpatient care in a manner that suits them best.

Assessment of glaucoma referral letter for quality and accuracy among patients referred to a tertiary eye care center.

Purpose: To assess the quality and accuracy of glaucoma referrals from ophthalmologist. Methods: Retrospective review of patients chart with referral letter to a tertiary glaucoma center between January and December 2017. Patients aged <16 years, lens-induced glaucoma, uveitic glaucoma, and glaucoma following retinal and corneal surgery were excluded. Results: A total of 184 patients referred by 55 ophthalmologists were included. Mean patient age (SD) was 57.8 ± 14 years. Intraocular pressure was not documented in the referral letter in 113 (61%) patients, gonioscopy in 174 (95%) patients, disc findings in 149 (81%) patients, and visual fields in 175 (95%) patients. Thirteen (37%) of the 35 patients referred as open angle glaucoma were found to have angle closure glaucoma. Pseudoexfoliation glaucoma was diagnosed in 29 (16%) patients, of which 18 were missed by the referring ophthalmologist. Conclusion: In our study >90% of referral letter did not have the essential parameters. A standard template for glaucoma referral is suggested, which will help the patient to get better transfer of care.

Are physicians in primary health care able to recognize pulmonary fibrosis?

Background: The early diagnosis of idiopathic pulmonary fibrosis (IPF) has become increasingly important due to evolving treatment options. IPF patients experience a significant delay in receiving an accurate diagnosis, thus delayed access to tertiary care is associated with higher mortality independently from disease severity. Objective: The aims were to evaluate whether there had been a delay in the referral process, and to determine whether the referring doctors had suspected IPF or other interstitial lung disease (ILD) already during the time of referral. Methods: Ninety-five referral letters of patients with IPF identified from the FinnishIPF registry were evaluated with respect to time of referral, referring unit, grounds for referral, symptoms, smoking status, occupational history, clinical examinations, co-morbidities, medication, radiological findings and lung function. Results: Fifty-nine percent of referral letters originated from primary public health care. The time from symptom onset to referral was reported in 60% of cases, mean time being 1.5 (0.8-2.3) (95%CI) years. The main reason for referral was a suspicion of interstitial lung disease (ILD) (63%); changes in chest X-ray were one reason for referring in 53% of cases. Lung auscultation was reported in 70% and inspiratory crackles in 52% of referral letters. Conclusions: Primary care doctors suspected lung fibrosis early in the course of disease. Lung auscultation and chest X-rays were the most common investigational abnormalities in the referrals. Providing general practitioners with more information of ILDs might shorten the delay from symptom onset to referral.

Older patient hospital admissions following primary care referral: the truth is in the referring.

BACKGROUND: Health information has a major role in the planning of future healthcare provision. With current reconfiguration and cost saving measures, further demands are being placed on acute hospitals. AIM: To examine the elderly admissions and the referral documentation of older patients admitted to a tertiary level hospital. METHODS: A retrospective analysis of primary care referral documentation for all acute admissions of patients over 75 years to University Hospital Limerick (UHL) over a 2-month period. Documentation was analysed on the basis of patient demographics, presenting complaint and referral source. Primary care referral documentation was then analysed on the basis of presenting complaint, patient demographics, referrer details, and the clinical information provided. RESULTS: Over the 2-month period there were a total of 381 elderly admissions through the Emergency department. The most common presenting complaint was with shortness of breath (21.5 %). 42.5 % of admissions were from a primary care setting. 31.1 % of referrals were typed and 47.0 % handwritten. Over 90 % of referrals contained the patient's name, date of birth and address. 98.7 % of referrals included a presenting complaint and 54 % included a past medical history. 20 % of referrals listed known drug allergies, while 9.3 % documented social history or baseline functional status. Referral letters from general practice and after-hour services were largely similar. CONCLUSIONS: Almost all primary care referrals included the required details as per recent HIQA guidelines. The further inclusion of optional information relating to patient social or functional status, which are of particular relevance to the older population may help patient management.

What proportion of adult allergy referrals to secondary care could be dealt with in primary care by a GP with special interest?

BACKGROUND: The concept of a General Practitioner with Special Interest (GPwSI) was first proposed in the 2000 National Health Service Plan, as a way of providing specialised treatment closer to the patient's home and reducing hospital waiting times. Given the patchy and inadequate provision of allergy services in the UK the introduction of GPwSIs might reduce the pressure on existing specialist services. OBJECTIVES: This study assessed what proportion of referrals to a specialist allergy clinic could be managed in a GPwSI allergy service with a predefined range of facilities and expertise (accurate diagnosis and management of allergy; skin prick testing; provision of advice on allergen avoidance; ability to assess suitability for desensitisation). METHODS: 100 consecutive GP referrals to a hospital allergy clinic were reviewed to determine whether patients could be seen in a community-based clinic led by a general practitioner with special interest (GPwSI) allergy. The documentation relating to each referral was independently assessed by three allergy specialists. The referrals were judged initially on the referral letter alone and then re-assessed with the benefit of information summarised in the clinic letter, to determine whether appropriate triage decisions could be made prospectively. The proportion of referrals suitable for a GPwSI was calculated and their referral characteristics identified. RESULTS: 29 % referrals were judged unanimously appropriate for management by a GPwSI and an additional 30 % by 2 of the 3 reviewers. 18 % referrals were unsuitable for a GPwSI service because of the complexity of the presenting problem, patient co-morbidity or the need for specialist knowledge or facilities. CONCLUSIONS AND CLINICAL RELEVANCE: At least a quarter, and possibly half, of allergy referrals to our hospital-based service could be dealt with in a GPwSI clinic, thereby diversifying the patient pathway, allowing specialist services to focus on more complex cases and reducing the waiting time for first appointments.

PROFSS: a screening tool for early identification of functional somatic symptoms.

OBJECTIVE: To develop and validate a brief screening tool for predicting functional somatic symptoms (FSS) based on clinical and non-clinical information from the general practitioner referral letter, and to assess its inter-rater reliability. METHODS: The derivation sample consisted of 357 consecutive patients referred to an internal outpatient clinic by their general practitioner. Referral letters were scored for candidate predictors for the main outcome measure, which was a final diagnosis of FSS made by the internist. Logistic regression identified the following independent predictors: type of symptoms, somatic and psychiatric comorbidity, absence of abnormal physical findings by the general practitioner, previous specialist consultation, and the use of illness terminology. Temporal validation was performed in a cohort of 94 consecutive patients in whom predictors were scored by two independent raters. RESULTS: In both the derivation and validation sample, the discriminatory power of the model was good with areas under the receiver operating characteristic curves of 0.84 (95%confidence interval: 0.80-0.88) after bootstrapping and 0.82 (95%confidence interval: 0.73-0.91), respectively. Calibration of the models was excellent in both samples and the interobserver agreement in the validation sample was very good (intraclass coefficient: 0.82 (95%confidence interval: 0.75-0.88)). Based on this model, we constructed the brief screening tool PROFSS (Predicted Risk Of Functional Somatic Symptoms). PROFSS identified patient groups with risks of FSS ranging from 17% (95%CI: 10-26%) to 92% (95%CI:86-96%). CONCLUSION: The presence of FSS can be predicted with the brief screening tool PROFSS, based on a limited set of items present in the general practitioner referral letter.