Assessing the Ability of a Large Language Model to Score Free-Text Medical Student Clinical Notes: Quantitative Study.

Background: Teaching medical students the skills required to acquire, interpret, apply, and communicate clinical information is an integral part of medical education. A crucial aspect of this process involves providing students with feedback regarding the quality of their free-text clinical notes. Objective: The goal of this study was to assess the ability of ChatGPT 3.5, a large language model, to score medical students' free-text history and physical notes. Methods: This is a single-institution, retrospective study. Standardized patients learned a prespecified clinical case and, acting as the patient, interacted with medical students. Each student wrote a free-text history and physical note of their interaction. The students' notes were scored independently by the standardized patients and ChatGPT using a prespecified scoring rubric that consisted of 85 case elements. The measure of accuracy was percent correct. Results: The study population consisted of 168 first-year medical students. There was a total of 14,280 scores. The ChatGPT incorrect scoring rate was 1.0%, and the standardized patient incorrect scoring rate was 7.2%. The ChatGPT error rate was 86%, lower than the standardized patient error rate. The ChatGPT mean incorrect scoring rate of 12 (SD 11) was significantly lower than the standardized patient mean incorrect scoring rate of 85 (SD 74; P=.002). Conclusions: ChatGPT demonstrated a significantly lower error rate compared to standardized patients. This is the first study to assess the ability of a generative pretrained transformer (GPT) program to score medical students' standardized patient-based free-text clinical notes. It is expected that, in the near future, large language models will provide real-time feedback to practicing physicians regarding their free-text notes. GPT artificial intelligence programs represent an important advance in medical education and medical practice.

Healthcare professional students' skills in sexual health communication and history taking: inter-rater reliability of standardized patients and faculty ratings in dar es Salaam, tanzania- a cross-sectional study.

BACKGROUND: Low- and middle-income countries face a disproportionate impact of sexual health problems compared to high-income countries. To address this situation proper interpersonal communication skills are essential for clinician to gather necessary information during medical history-taking related to sexual health. This study aimed to evaluate the interrater reliability of ratings on sexual health-related interpersonal communication and medical history-taking between SPs and trained HCP faculty for health care professional students. METHODS: We conducted a cross-sectional comparative study to evaluate the interrater reliability of ratings for sexual health-related interpersonal communication and medical history-taking. The data were collected from medical and nursing students at Muhimbili University of Health and Allied Sciences, who interviewed 12 Standardized Patients (SPs) presenting with sexual health issues. The video-recorded interviews rated by SPs, were compared to the one rated by 5 trained Health Care Professional (HCP) faculty members. Inter-rater reliability was evaluated using percent agreement (PA) and kappa statistics (κ). RESULTS: A total of 412 students (mean age 24) were enrolled in the study to conduct interviews with two SPs presenting with sexual health concerns. For interpersonal communication (IC), the overall median agreement between raters was slight (κ2 0.0095; PA 48.9%) while the overall median agreement for medical sexual history-taking was deemed fair (κ2 0.139; PA 75.02%). CONCLUSION: The use of SPs for training and evaluating medical and nursing students in Tanzania is feasible only if they undergo proper training and have sufficient time for practice sessions, along with providing feedback to the students.

Virtual patients versus standardized patients for improving clinical reasoning skills in ophthalmology residents. A randomized controlled trial.

BACKGROUND: History taking and clinical reasoning are important skills that require knowledge, cognition and meta-cognition. It is important that a trainee must experience multiple encounters with different patients to practice these skills. However, patient safety is also important, and trainees are not allowed to handle critically ill patients. To address this issue, a randomized controlled trial was conducted to determine the effectiveness of using Virtual Patients (VP) versus Standardized Patients (SP) in acquiring clinical reasoning skills in ophthalmology postgraduate residents. METHODS: Postgraduate residents from two hospitals in Lahore, Pakistan, were randomized to either the VP group or the SP group and were exposed to clinical reasoning exercise via the VP or SP for 30 min after the pretest. This was followed by a posttest. One month after this activity, a follow-up posttest was conducted. The data were collected and analysed using IBM-SPSS version 25. Repeated measures ANOVA was used to track the effect of learning skills over time. RESULTS: The mean age of the residents was 28.5 ± 3 years. The male to female ratio was 1:1.1. For the SP group, the mean scores were 12.6 ± 3.08, 16.39 ± 3.01 and 15.39 ± 2.95, and for the VP group, the mean scores were 12.7 ± 3.84, 16.30 ± 3.19 and 15.65 ± 3.18 for the pretest, posttest and follow-up posttest, respectively (p value < 0.00). However, the difference between the VP and SP groups was not statistically significant (p = 0.896). Moreover, there was no statistically significant difference between the VP and SP groups regarding the retention of clinical reasoning ability. In terms of learning gain, compared with the VP group, the SP group had a score of 51.46% immediately after clinical reasoning exercise as compared to VP group, in which it was 49.1%. After one month, it was 38.01 in SP and 40.12% in VP group. CONCLUSION: VPs can be used for learning clinical reasoning skills in postgraduate ophthalmology residents in a safe environment. These devices can be used repeatedly without any risk to the real patient. Although similarly useful, SP is limited by its nonavailability for repeated exercises.

Criteria to evaluate graduate nurse proficiencies in obtaining a health history and perform physical assessment in simulation-based education: A narrative review.

BACKGROUND: Simulation is a technique being used increasingly in healthcare education which offers opportunities to evaluate nursing proficiencies. The use of valid and reliable instruments is recognised as the foundation for a robust assessment, however competency-based health assessment courses for graduate nurses can consequently become reductionist in measuring proficiencies. OBJECTIVE: The specific review question was: In simulation-based education, what are the criteria that evaluate graduate nursing student's competence in obtaining a health history and performance of patient assessment? METHODS: Eleven studies were included in the review. Papers were critically appraised with The Joanna Briggs Institute quasi-experimental studies checklist. Bloom's taxonomy was used to structure this narrative review. RESULTS: Seven papers evaluated cognition through questionnaires and two papers used a Likert-scale to determine self-perceived knowledge. Six papers evaluated psychomotor skills with a behavioural checklist. Diversity of application was factored into the studies when testing affective skills. Three papers used Likert-scales to evaluate preparedness, six papers used Likert-scales to evaluate self-confidence and one used a Likert-scale to evaluate autonomy. Three papers used a checklist to evaluate professionalism. Four papers used faculty member/ standardised patient feedback. CONCLUSION: Reductionist evaluation instruments create a barrier when evaluating competency. The limited validity and reliability of assessment instruments in simulation, as well as the lack of standardisation of affective skills assessment, presents a challenge in simulation research. Affective skills encompass attitudes, behaviours and communication abilities, which pose a significant challenge for standardised assessments due to their subjective nature. This review of the simulation literature highlights a lack of robustness in the evaluation of the affective domain. This paper proposes that simulation assessment instruments should include the standardisation of affective domain proficiencies such as: adaptation to patients' cognitive function, ability to interpret and synthesise relevant information, ability to demonstrate clinical judgement, readiness to act, recognition of professional limitations and faculty/standardised-simulated patient feedback. The incorporation of the affective domain in standardised assessment instruments is important to ensure comprehensive assessment of simulation particularly in the development of health history and physical assessment proficiencies. Attention to all of the domains in Blooms taxonomy during simulation assessment has the potential to better prepare professionals for the patient care setting.

Lost in translation: Unveiling medical students' untold errors of medical history documentation.

OBJECTIVE: The accurate documentation of a medical history interview is an important goal in medical education. As students' documentation of medical history interviews is mostly decentralised on the wards, a systematic assessment of documentation quality is missing. We therefore evaluated the extent of details missed in students' medical history reports in a standardised setting. METHODS: In this prospective, observational study, 123 of 380 students (32.4%) participated in an Objective Structured Clinical Examination (OSCE) regarding history taking and documentation. Based on the interviews and nine deductively selected main categories, a categorical system was established using a summarising qualitative content analysis. The items in the transcripts (defined as ground truth) and in students' reports were labelled and assigned to the correct subcategory. The ground truth and students' reports were compared to quantify students' documentation completeness. RESULTS: Next to the nine deductively selected main categories, 61 subcategories were defined. A total of 8943 items were labelled in the 123 interview transcripts (ground truth), compared with 5870 items labelled in students' reports (65.6% completeness of students' reports compared with ground truth). The main category personal details overlapped with 94.2% between students' report and ground truth in contrast to the main category with the highest discrepancy, allergy, with 41.1% overlap. Pertinent negative items and non-numerical quantifications were often missed. CONCLUSIONS: Medical students show incomplete documentation of medical history interviews. Therefore, accurate documentation should be taught as an important goal in medical education.

The Military Health History: An Educational Initiative to Improve Veteran Healthcare.

INTRODUCTION: As a result of the nature of military service, veterans are a unique patient population with many special health considerations. For various reasons, measures are often not taken by clinicians to address such special considerations. This results in a healthcare disparity for veterans first described by Dr. Jeffrey Brown in 2012. To address this disparity, we introduced "the military health history" to third-year medical students at a large medical school in the southeastern United States. Our objective was to assess the effectiveness of this educational intervention and determine its potential role in creating a future in which veteran healthcare is of the highest quality. MATERIALS AND METHODS: This study was approved as a quality assurance/quality improvement project by both Louisiana State University Health Science Center and Southeast Louisiana Veteran Healthcare System IRB offices. A short lecture outlining the components of the military health history was presented to 186 third-year medical students. The students were given the opportunity to answer five survey questions before and after the lecture. These questions assessed the students' current confidence performing a military health history, perceived importance of doing so, and likelihood of future implementation. To determine useful retention of the lecture material, the post-lecture survey was readministered to the same population sample 6 months after the lecture. A series of repeated measures analyses of variance were conducted to examine changes in mean levels of confidence, importance, and likelihood of ascertaining military history during a patient encounter at pre- and post-presentation as well as at the 6 month follow-up. RESULTS: Results revealed a significant effect of time on importance, Wilks' Lambda = 0.74, F (2, 87) = 15.41, P < 0.001; confidence, Wilks' Lambda = 0.61, F (2, 87) = 27.58, P < 0.001; and likelihood of ascertaining a military history during a future patient encounter, Wilks' Lambda = 0.46, F (2, 88) = 50.58, P < 0.001. Results are demonstrated in detail in Table I of the manuscript. CONCLUSION: The lecture resulted in a statistically significant increase over 6 months in both the likelihood and confidence parameters. The team believes that this result indicates that the students demonstrated useful retention of the lecture material. Our hope is that these students continue to employ the military health history throughout their years of clinical work. In the future, we plan to survey veterans immediately following Veterans Health Administration clinic visits with members of our study population to assess the patient's perceived benefit of the military health history. The team will continue to investigate ways in which military health curricula can be implemented in undergraduate medical education.

[Step by step - A diagnostic approach to bleeding disorders]. / Abklärung erhöhter Blutungsneigung ­ Schritt für Schritt.

The main symptom of hemorrhagic diathesis is an increased bleeding tendency. Due to the subjectivity of various features of the bleeding history, unclarity of the family history, and an individualization of the extent of diagnostic the evaluation of a suspected bleeding disorder represents a challenging endeavour in hematology. Hemorrhagic diathesis can be divided into the following sub-categories: disorders in primary hemostasis (e. g. von Willebrand disease, different causes of thrombocytopenia), secondary hemostasis (e. g. hemophilia A and B, Vitamin K deficiency) and fibrinolysis, and in connective tissue or vascular formation. This article reviews available diagnostic methods for bleeding disorders, from structured patient history to highly specialized laboratory diagnosis.

Determination of thresholds of risk in women at average risk of breast cancer to personalize the organized screening program.

In France, more than 10 million women at "average" risk of breast cancer (BC), are included in the organized BC screening. Existing predictive models of BC risk are not adapted to the French population. Thus, we set up a new score in the French Hérault region and looked for subgroups at a graded level of risk in women at "average" risk. We recruited a retrospective cohort of women, aged 50 to 60, who underwent the organized BC screening, and included 2241 non-cancer women and 527 who developed a BC during a 12-year follow-up period (2006-2018). The risk factors identified were high breast density (ACR BI-RADS grading)(B vs A: HR = 1.41, 95%CI [1.05; 1.9], p = 0.023; C vs A: HR = 1.65 [1.2; 2.27], p = 0.02 ; D vs A: HR = 2.11 [1.25;3.58], p = 0.006), a history of maternal breast cancer (HR = 1.61 [1.24; 2.09], p < 0.001), and socioeconomic difficulties (HR 1.23 [1.09; 1.55], p = 0.003). While early menopause (HR = 0.36 [0.13; 0.99], p = 0.003) and an age at menarche after 12 years (HR = 0.77 [0.63; 0.95], p = 0.047) were protective factors. We identified 3 groups at risk: lower, average, and higher, respectively. A low threshold was characterized at 1.9% of 12-year risk and a high threshold at 4.5% 12-year risk. Mean 12-year risks in the 3 groups of risk were 1.37%, 2.68%, and 5.84%, respectively. Thus, 12% of women presented a level of risk different from the average risk group, corresponding to 600,000 women involved in the French organized BC screening, enabling to propose a new strategy to personalize the national BC screening. On one hand, for women at lower risk, we proposed to reduce the frequency of mammograms and on the other hand, for women at higher risk, we suggested intensifying surveillance.

Dyspareunia in Women.

Dyspareunia is recurrent or persistent pain with sexual intercourse that causes distress. It affects approximately 10% to 20% of U.S. women. Dyspareunia may be superficial, causing pain with attempted vaginal insertion, or deep. Women with sexual pain are at increased risk of sexual dysfunction, relationship distress, diminished quality of life, anxiety, and depression. Because discussing sexual issues may be uncomfortable, clinicians should create a safe and welcoming environment when taking a sexual history, where patients describe the characteristics of the pain (e.g., location, intensity, duration). Physical examination of the external genitalia includes visual inspection and sequential pressure with a cotton swab, assessing for focal erythema or pain. A single-digit vaginal examination may identify tender pelvic floor muscles, and a bimanual examination can assess for uterine retroversion and pelvic masses. Common diagnoses include vulvodynia, inadequate lubrication, vaginal atrophy, postpartum causes, pelvic floor dysfunction, endometriosis, and vaginismus. Treatment is focused on the cause and may include lubricants, pelvic floor physical therapy, topical analgesics, vaginal estrogen, cognitive behavior therapy, vaginal dilators, modified vestibulectomy, or onabotulinumtoxinA injections.

Diagnostic Accuracy of Patient History in the Diagnosis of Hip-Related Pain: A Systematic Review.

OBJECTIVE: To investigate the diagnostic accuracy of patient history associated with hip pain. DATA SOURCES: A systematic, computerized search of electronic databases (PubMed, MEDLINE, Cumulative Index of Nursing and Allied Health Literature, and Web of Science), a search of the gray literature, and review of the primary author's personal library was performed. Hip-specific search terms were combined with diagnostic accuracy and subjective or self-report history-based search terms using the Boolean operator "AND." STUDY SELECTION: This systematic review was conducted and reported according to the protocol outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The inclusion criteria were: (1) patients with hip pain; (2) the statistical association of at least 1 patient history item was reported; (3) study designs appropriate for diagnostic accuracy; (4) adults aged ≥18 years; (5) written in English; and (6) used an acceptable reference standard for diagnosed hip pathology. Titles and abstracts of all database-captured citations were independently screened by at least 2 reviewers. DATA EXTRACTION: Two reviewers independently extracted information and data regarding author, year, study population, study design, criterion standard, and strength of association statistics associated with the subjective findings. DATA SYNTHESIS: For hip osteoarthritis (OA), a family history of OA (positive likelihood ratio [+LR], 2.13), history of knee OA (+LR, 2.06), report of groin or anterior thigh pain (+LR, 2.51-3.86), self-reported limitation in range of motion of 1 or both hips (+LR, 2.87), constant low back pain or buttock pain (+LR, 6.50), groin pain on the same side (+LR, 3.63), and a screening questionnaire (+LR, 3.87-13.29) were the most significant findings. For intra-articular hip pathology, crepitus (+LR, 3.56) was the most significant finding. CONCLUSIONS: Patient history plays a key role in differential diagnosis of hip pain and in some cases can be superior to objective tests and measures.

[Machine learning is unlikely to improve the diagnostic performance of medical history taking]. / Betere zorg door anamnesestandaard.

Use of machine learning has been proposedtoimprovethediagnostic performance of medicalhistorytaking, whichwould first have tobestandardized. Thiscommentary reviews theoretical, practical andethicalconsiderationswithregardtothisproposal.

[Clinical history in times of big data; a plea for a standard for the structured recording of the clinical history]. / Anamnese in tijden van big data.

Clinical decision support systems to aid the clinician in making a correct diagnosis will only succeed if data from the clinical history are taken into account. However, currently, very little is known on diagnostic test characteristics of specific symptoms, let alone of a pattern of several symptoms with all their cardinal features. We plead for the nation-wide introduction of a standard for the structured recording of the clinical history. To allow for such structured recording, user interfaces of electronic healthcare records must become far more user-friendly. Furthermore, scribes may be used, or, ideally, a digital scribe, a computer application that records the conversation between healthcare professional and patient and creates an automated summary. So far, to our knowledge, no digital scribe encompassing the entire patient history has been implemented into medical practice. We are currently trying to develop such a digital scribe.

Recommended Musculoskeletal and Sports Medicine Model Curriculum for Emergency Medicine Residency Training.

ABSTRACT: Musculoskeletal and sports medicine conditions are common in the emergency department (ED). Emergency physicians may not be receiving adequate education to achieve clinical competency in musculoskeletal medicine during residency training. This article aims to provide a standardized musculoskeletal and sports medicine curriculum for emergency medicine training. Broad curriculum goals include proficiency in evaluating and managing patients presenting to the ED with acute and chronic musculoskeletal complaints and other medical conditions related to or affected by physical exertion, sports participation, or environmental exposure. Specific objectives focus on knowledge of these disorders, physical examination skills, procedural skills including musculoskeletal ultrasound, appropriate consultation and referral, and patient education for these conditions. Educational methods will consist of didactics; online self-directed learning modules; simulation; and supervised clinical experiences in the ED, primary care sports medicine clinics, and orthopedic clinics if available. Curriculum implementation is expected to vary across programs due to differences in residency program structure and resources.

Gynecologic Considerations for the Urologic Surgeon.

While gynecologic malignancy is uncommon in women with conditions such as pelvic organ prolapse and bladder cancer, urologists should be acquainted with the relevant gynecologic literature as it pertains to their surgical care of female patients. While taking the patient history, urologists should be aware of prior cervical cancer screening and ask about vaginal bleeding, which can be a sign of uterine cancer. Urologic surgeons should also discuss the role of concomitant prophylactic oophorectomy and/or salpingectomy for ovarian cancer risk reduction at the time of pelvic surgery. An understanding of basic tests, such as a transvaginal sonogram, can help urologists provide comprehensive care.

[Pneumo-Quest: A standardised self-questionnaire to be completed at home before a first appointment at a respiratory clinic]. / Pneumo-Quest, auto-questionnaire standardisé à compléter au domicile avant une première consultation en pneumologie : étude de validation.

The Pneumo-Quest self-questionnaire was developed to standardize the practice of recollection when welcoming a new patient. It consists of 82 main questions and 34 subsidiary questions to be completed at home by the patients before their first visit to a pulmonologist. This evaluation was carried out on the basis of 137 returned questionnaires. The feasibility (main criterion) was good with 93±5% of the questions answered and an average completion time of 15.1±9.8minutes (mean±SD). The reliability of the responses (secondary criterion) was good with the agreement between the patient's response and the doctor's opinion being excellent or good for the majority of medical histories and treatments, as evidenced by the high values of the kappa coefficient (>0.90; <0.90; <0.75). Patient and physician perception of the questionnaire was good with 99% and 90% positive ratings, respectively. The use of the questionnaire was unhelpful in the course of the consultation in only 2% of cases. Doctors found the tool useful for obtaining a comprehensive history in 87% of cases and patients declared that it helped them "forgot nothing" in 93% of the cases. The questionnaire helped the doctor to identify the patient's problems rapidly in 71% of cases and saved time in 64%. These positive results encourage a wide dissemination of the questionnaire (www.pneumo-quest.com).

Assessing the Reliability of Reported Medical History in Older Adults.

BACKGROUND: Age-associated increases in medical complexity, frailty, and cognitive impairment may compromise reliable reporting of medical history. OBJECTIVE: To evaluate the influence of increasing age and cognitive impairment on concordance between reported history of stroke and cerebral infarction, and reported history of diabetes and elevated hemoglobinA1c in community-dwelling older adults. METHODS: The association between participant-specific factors and accurate reporting of stroke or diabetes was evaluated using multivariable logistic regression in 1,401 participants enrolled in longitudinal studies of memory and aging, including 425 participants with dementia (30.3%). Stroke and diabetes were selected as index variables as gold standard measures of both were obtained in all participants: magnetic resonance neuroimaging for cerebral infarcts and hemoglobinA1c (≥6.5%) for diabetes. RESULTS: Concordance between reported history of stroke and imaging-confirmed cerebral infarction was low (sensitivity: 17.4%, 8/46; specificity: 97.9%, 799/816). Small infarcts were strongly associated with inaccurate reporting (OR = 265.8; 95% CI: 86.2, 819.4), suggesting that occult/silent infarcts contributed to discordant reporting. Reporting accuracy was higher concerning diabetes (sensitivity: 83.5%, 147/176; specificity: 96.2%, 1100/1143). A history of hypertension (OR = 2.3; 95% CI: 1.3, 4.2), higher hemoglobinA1c (OR = 1.9; 95% CI: 1.5, 2.4), and hemoglobinA1c compatible with impaired glucose tolerance (OR = 3.1; 95% CI 1.8, 5.3) associated with increased odds of discordant reporting. Cognitive impairment and increased age were not independently associated with reliable reporting. CONCLUSION: Factors beyond advancing age and cognitive impairment appear to drive discordance in reported medical history in older participants. Objective testing for cerebral infarcts or diabetes should be performed when relevant to diagnostic or therapeutic decisions in clinical and research settings.

[How complete is the Germany-wide standardised medication list ("Bundeseinheitlicher Medikationsplan")? An analysis at hospital admission.] / Wie vollständig ist der Bundeseinheitliche Medikationsplan? Eine Analyse bei Krankenhausaufnahme.

BACKGROUND: At inpatient admission, the timeliness and completeness of the Germany-wide standardised medication list ("Bundeseinheitlicher Medikationsplan") often seems inappropriate. It is also unclear which characteristics of the lists increase the probability of discrepancies. METHODS: A total of 100 medication lists of elective patients of a surgical clinic were retrospectively evaluated with regard to potential discrepancies compared to the standardised medication reconciliation. The discprepancies were assigned to 7 categories: drug taken is missing on the list, drug on the list is no longer taken, strength or dosage is missing at the list or is incorrect, or the documented dosage form is different. Advice on patient safety, involved drugs and dosage forms were also recorded. Multivariate analyses were used to investigate the influence of the timeliness, number of drugs and issuing medical specialty of the lists on the type and number of discrepancies. RESULTS: Compared to the medication reconciliation, 78 % (78/100) of the lists showed discrepancies. A total of 226 deviations (2.3 ±â€Š0.6 deviations/list) were documented. Most often, a drug was missing from the list (n = 103). Of all recommendations, 64 % (83/177) concerned the perioperative management of anticoagulants (n = 55) and antidiabetics (n = 28), corresponding to 62 % (62/100) of the lists. In the multivariate analysis, only the risk of incorrect information on strength and dosage increased significantly with the age of the lists (p = 0.047) and was more than twice as high when the list was more than one month old. CONCLUSIONS: The timeliness, completeness and aspects of patient safety must be comprehensively validated. Medication lists that are older than 1 month should be checked particularly critically with regard to information on strength and dosage and the plan should be updated accordingly at regular intervals.