Disparity in Medicare Reimbursement Between Female and Male Vitreoretinal Surgeons.

Purpose: To quantify the Medicare reimbursement disparity between female and male vitreoretinal surgeons. Methods: Reimbursement reports were obtained from the US Center for Medicare and Medicaid Services from 2013 through 2020, which detail all Medicare Part B services. A vitreoretinal surgeon was defined as any provider with at least 10 charges of a Healthcare Common Procedure Coding System code related to vitrectomy or retinal detachment repair. Providers were grouped by sex, and the average total reimbursement rate and additional secondary statistics to quantify the reimbursement disparity were identified. Results: On average, female vitreoretinal surgeons were reimbursed 65% that of their male counterparts in 2020, $1.66 million to $2.56 million. The percentage of the average male vitreoretinal specialist's total reimbursement that the average female vitreoretinal specialist received decreased 8.8% from 2013 to 2020, from 73.8% to 65.0%. Conclusions: The reimbursement that the average female vitreoretinal surgeon receives from Medicare is only two thirds that of the average male vitreoretinal surgeon. In addition, there was no identifiable improvement in this disparity over the study period. Further efforts must be taken to establish concerted efforts to improve the reimbursement disparity and to identify the systematic inequities that led to its presence in the first place.

The Scope of Multimorbidity in Family Medicine: Identifying Age Patterns Across the Lifespan.

INTRODUCTION: Multimorbidity rates are both increasing in prevalence across age ranges, and also increasing in diagnostic importance within and outside the family medicine clinic. Here we aim to describe the course of multimorbidity across the lifespan. METHODS: This was a retrospective cohort study across 211,953 patients from a large northeastern health care system. Past medical histories were collected in the form of ICD-10 diagnostic codes. Rates of multimorbidity were calculated from comorbid diagnoses defined from the ICD10 codes identified in the past medical histories. RESULTS: We identify 4 main age groups of diagnosis and multimorbidity. Ages 0 to 10 contain diagnoses which are infectious or respiratory, whereas ages 10 to 40 are related to mental health. From ages 40 to 70 there is an emergence of alcohol use disorders and cardiometabolic disorders. And ages 70 to 90 are predominantly long-term sequelae of the most common cardiometabolic disorders. The mortality of the whole population over the study period was 5.7%, whereas the multimorbidity with the highest mortality across the study period was Circulatory Disorders-Circulatory Disorders at 23.1%. CONCLUSION: The results from this study provide a comparison for the presence of multimorbidity within age cohorts longitudinally across the population. These patterns of comorbidity can assist in the allocation to practice resources that will best support the common conditions that patients need assistance with, especially as the patients transition between pediatric, adult, and geriatric care. Future work examining and comparing multimorbidity indices is warranted.

Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models.

BACKGROUND: Natural language processing (NLP) tools including recently developed large language models (LLMs) have myriad potential applications in medical care and research, including the efficient labeling and classification of unstructured text such as electronic health record (EHR) notes. This opens the door to large-scale projects that rely on variables that are not typically recorded in a structured form, such as patient signs and symptoms. OBJECTIVES: This study is designed to acquaint the emergency medicine research community with the foundational elements of NLP, highlighting essential terminology, annotation methodologies, and the intricacies involved in training and evaluating NLP models. Symptom characterization is critical to urinary tract infection (UTI) diagnosis, but identification of symptoms from the EHR has historically been challenging, limiting large-scale research, public health surveillance, and EHR-based clinical decision support. We therefore developed and compared two NLP models to identify UTI symptoms from unstructured emergency department (ED) notes. METHODS: The study population consisted of patients aged ≥ 18 who presented to an ED in a northeastern U.S. health system between June 2013 and August 2021 and had a urinalysis performed. We annotated a random subset of 1250 ED clinician notes from these visits for a list of 17 UTI symptoms. We then developed two task-specific LLMs to perform the task of named entity recognition: a convolutional neural network-based model (SpaCy) and a transformer-based model designed to process longer documents (Clinical Longformer). Models were trained on 1000 notes and tested on a holdout set of 250 notes. We compared model performance (precision, recall, F1 measure) at identifying the presence or absence of UTI symptoms at the note level. RESULTS: A total of 8135 entities were identified in 1250 notes; 83.6% of notes included at least one entity. Overall F1 measure for note-level symptom identification weighted by entity frequency was 0.84 for the SpaCy model and 0.88 for the Longformer model. F1 measure for identifying presence or absence of any UTI symptom in a clinical note was 0.96 (232/250 correctly classified) for the SpaCy model and 0.98 (240/250 correctly classified) for the Longformer model. CONCLUSIONS: The study demonstrated the utility of LLMs and transformer-based models in particular for extracting UTI symptoms from unstructured ED clinical notes; models were highly accurate for detecting the presence or absence of any UTI symptom on the note level, with variable performance for individual symptoms.

Ophthalmic care may not align with patient need: An analysis on state-wide patient needs and provider density between 2008 and 2022.

PURPOSE: This study aims to assess the extent to which the demand for ophthalmologic care among patients at the state level is reflected in Google Trends data, serving as an indicator of patient desire in ophthalmology. METHODS: For each state, patient interest in ophthalmologic care was estimated using the Google Trends resource measuring web search and YouTube search rates for multiple ophthalmologic terms. We compared the change in search for ophthalmologic terms over time and used ordinary least squares regression to evaluate whether search interest for ophthalmologic terms was able to predict the rate of practicing ophthalmologists in each state. We also compare the changing rates of searches across the web and YouTube to evaluate the resources patients are most likely to utilize. RESULTS: From 2008 to 2022, web search rates for general ophthalmology related terms increased by 43.98%, while search interest for retinal specific terms increased by 19.51%. YouTube specific results for general ophthalmology terms increased by 55.83% while search for retinal terms fell by 58.48%. Ophthalmologic and retinal specific search interest was not significantly associated with either outcome. CONCLUSIONS: Our findings suggest that patient information needs, demographic elements, and the educational backgrounds of residents and fellows - those important factors - are surprisingly poorly correlated with ophthalmology provider density. Furthermore, we observed no noteworthy correlation between the search interest in ophthalmology and the overall density of ophthalmologists or retinal specialists. This implies that there is a pressing need to explore and implement strategies aimed at better aligning these influencing factors the choices made by ophthalmologists in selecting their practice locations to bridge the gap between healthcare availability and public interest.

Identifying incarceration status in the electronic health record using large language models in emergency department settings.

Background: Incarceration is a significant social determinant of health, contributing to high morbidity, mortality, and racialized health inequities. However, incarceration status is largely invisible to health services research due to inadequate clinical electronic health record (EHR) capture. This study aims to develop, train, and validate natural language processing (NLP) techniques to more effectively identify incarceration status in the EHR. Methods: The study population consisted of adult patients (≥ 18 y.o.) who presented to the emergency department between June 2013 and August 2021. The EHR database was filtered for notes for specific incarceration-related terms, and then a random selection of 1,000 notes was annotated for incarceration and further stratified into specific statuses of prior history, recent, and current incarceration. For NLP model development, 80% of the notes were used to train the Longformer-based and RoBERTa algorithms. The remaining 20% of the notes underwent analysis with GPT-4. Results: There were 849 unique patients across 989 visits in the 1000 annotated notes. Manual annotation revealed that 559 of 1000 notes (55.9%) contained evidence of incarceration history. ICD-10 code (sensitivity: 4.8%, specificity: 99.1%, F1-score: 0.09) demonstrated inferior performance to RoBERTa NLP (sensitivity: 78.6%, specificity: 73.3%, F1-score: 0.79), Longformer NLP (sensitivity: 94.6%, specificity: 87.5%, F1-score: 0.93), and GPT-4 (sensitivity: 100%, specificity: 61.1%, F1-score: 0.86). Conclusions: Our advanced NLP models demonstrate a high degree of accuracy in identifying incarceration status from clinical notes. Further research is needed to explore their scaled implementation in population health initiatives and assess their potential to mitigate health disparities through tailored system interventions.

Correction: How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment.

[This corrects the article DOI: 10.2196/45312.].

Dementia risk analysis using temporal event modeling on a large real-world dataset.

The objective of the study is to identify healthcare events leading to a diagnosis of dementia from a large real-world dataset. This study uses a data-driven approach to identify temporally ordered pairs and trajectories of healthcare codes in the electronic health record (EHR). This allows for discovery of novel temporal risk factors leading to an outcome of interest that may otherwise be unobvious. We identified several known (Down syndrome RR = 116.1, thiamine deficiency RR = 76.1, and Parkinson's disease RR = 41.1) and unknown (Brief psychotic disorder RR = 68.6, Toxic effect of metals RR = 40.4, and Schizoaffective disorders RR = 40.0) factors for a specific dementia diagnosis. The associations with the greatest risk for any dementia diagnosis were found to be primarily related to mental health (Brief psychotic disorder RR = 266.5, Dissociative and conversion disorders RR = 169.8), or neurologic conditions or procedures (Dystonia RR = 121.9, Lumbar Puncture RR = 119.0). Trajectory and clustering analysis identified factors related to cerebrovascular disorders, as well as diagnoses which increase the risk of toxic imbalances. The results of this study have the ability to provide valuable insights into potential patient progression towards dementia and improve recognition of patients at risk for developing dementia.

Computational phenotypes for patients with opioid-related disorders presenting to the emergency department.

OBJECTIVE: We aimed to discover computationally-derived phenotypes of opioid-related patient presentations to the ED via clinical notes and structured electronic health record (EHR) data. METHODS: This was a retrospective study of ED visits from 2013-2020 across ten sites within a regional healthcare network. We derived phenotypes from visits for patients ≥18 years of age with at least one prior or current documentation of an opioid-related diagnosis. Natural language processing was used to extract clinical entities from notes, which were combined with structured data within the EHR to create a set of features. We performed latent dirichlet allocation to identify topics within these features. Groups of patient presentations with similar attributes were identified by cluster analysis. RESULTS: In total 82,577 ED visits met inclusion criteria. The 30 topics were discovered ranging from those related to substance use disorder, chronic conditions, mental health, and medical management. Clustering on these topics identified nine unique cohorts with one-year survivals ranging from 84.2-96.8%, rates of one-year ED returns from 9-34%, rates of one-year opioid event 10-17%, rates of medications for opioid use disorder from 17-43%, and a median Carlson comorbidity index of 2-8. Two cohorts of phenotypes were identified related to chronic substance use disorder, or acute overdose. CONCLUSIONS: Our results indicate distinct phenotypic clusters with varying patient-oriented outcomes which provide future targets better allocation of resources and therapeutics. This highlights the heterogeneity of the overall population, and the need to develop targeted interventions for each population.

The Role of Large Language Models in Medical Education: Applications and Implications.

Large language models (LLMs) such as ChatGPT have sparked extensive discourse within the medical education community, spurring both excitement and apprehension. Written from the perspective of medical students, this editorial offers insights gleaned through immersive interactions with ChatGPT, contextualized by ongoing research into the imminent role of LLMs in health care. Three distinct positive use cases for ChatGPT were identified: facilitating differential diagnosis brainstorming, providing interactive practice cases, and aiding in multiple-choice question review. These use cases can effectively help students learn foundational medical knowledge during the preclinical curriculum while reinforcing the learning of core Entrustable Professional Activities. Simultaneously, we highlight key limitations of LLMs in medical education, including their insufficient ability to teach the integration of contextual and external information, comprehend sensory and nonverbal cues, cultivate rapport and interpersonal interaction, and align with overarching medical education and patient care goals. Through interacting with LLMs to augment learning during medical school, students can gain an understanding of their strengths and weaknesses. This understanding will be pivotal as we navigate a health care landscape increasingly intertwined with LLMs and artificial intelligence.

Predicting relations between SOAP note sections: The value of incorporating a clinical information model.

Physician progress notes are frequently organized into Subjective, Objective, Assessment, and Plan (SOAP) sections. The Assessment section synthesizes information recorded in the Subjective and Objective sections, and the Plan section documents tests and treatments to narrow the differential diagnosis and manage symptoms. Classifying the relationship between the Assessment and Plan sections has been suggested to provide valuable insight into clinical reasoning. In this work, we use a novel human-in-the-loop pipeline to classify the relationships between the Assessment and Plan sections of SOAP notes as a part of the n2c2 2022 Track 3 Challenge. In particular, we use a clinical information model constructed from both the entailment logic expected from the aforementioned Challenge and the problem-oriented medical record. This information model is used to label named entities as primary and secondary problems/symptoms, events and complications in all four SOAP sections. We iteratively train separate Named Entity Recognition models and use them to annotate entities in all notes/sections. We fine-tune a downstream RoBERTa-large model to classify the Assessment-Plan relationship. We evaluate multiple language model architectures, preprocessing parameters, and methods of knowledge integration, achieving a maximum macro-F1 score of 82.31%. Our initial model achieves top-2 performance during the challenge (macro-F1: 81.52%, competitors' macro-F1 range: 74.54%-82.12%). We improved our model by incorporating post-challenge annotations (S&O sections), outperforming the top model from the Challenge. We also used Shapley additive explanations to investigate the extent of language model clinical logic, under the lens of our clinical information model. We find that the model often uses shallow heuristics and nonspecific attention when making predictions, suggesting language model knowledge integration requires further research.

How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment.

BACKGROUND: Chat Generative Pre-trained Transformer (ChatGPT) is a 175-billion-parameter natural language processing model that can generate conversation-style responses to user input. OBJECTIVE: This study aimed to evaluate the performance of ChatGPT on questions within the scope of the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 exams, as well as to analyze responses for user interpretability. METHODS: We used 2 sets of multiple-choice questions to evaluate ChatGPT's performance, each with questions pertaining to Step 1 and Step 2. The first set was derived from AMBOSS, a commonly used question bank for medical students, which also provides statistics on question difficulty and the performance on an exam relative to the user base. The second set was the National Board of Medical Examiners (NBME) free 120 questions. ChatGPT's performance was compared to 2 other large language models, GPT-3 and InstructGPT. The text output of each ChatGPT response was evaluated across 3 qualitative metrics: logical justification of the answer selected, presence of information internal to the question, and presence of information external to the question. RESULTS: Of the 4 data sets, AMBOSS-Step1, AMBOSS-Step2, NBME-Free-Step1, and NBME-Free-Step2, ChatGPT achieved accuracies of 44% (44/100), 42% (42/100), 64.4% (56/87), and 57.8% (59/102), respectively. ChatGPT outperformed InstructGPT by 8.15% on average across all data sets, and GPT-3 performed similarly to random chance. The model demonstrated a significant decrease in performance as question difficulty increased (P=.01) within the AMBOSS-Step1 data set. We found that logical justification for ChatGPT's answer selection was present in 100% of outputs of the NBME data sets. Internal information to the question was present in 96.8% (183/189) of all questions. The presence of information external to the question was 44.5% and 27% lower for incorrect answers relative to correct answers on the NBME-Free-Step1 (P<.001) and NBME-Free-Step2 (P=.001) data sets, respectively. CONCLUSIONS: ChatGPT marks a significant improvement in natural language processing models on the tasks of medical question answering. By performing at a greater than 60% threshold on the NBME-Free-Step-1 data set, we show that the model achieves the equivalent of a passing score for a third-year medical student. Additionally, we highlight ChatGPT's capacity to provide logic and informational context across the majority of answers. These facts taken together make a compelling case for the potential applications of ChatGPT as an interactive medical education tool to support learning.

Patterns of Care Partner Communication for Persons Living with Dementia in the Emergency Department.

Introduction: Nearly half of all persons living with dementia (PLwD) will visit the emergency department (ED) in any given year and ED visits by PLwD are associated with short-term adverse outcomes. Care partner engagement is critical in the care of PLwD, but little is known about their patterns of communication with ED clinicians. Methods: We performed a retrospective electronic health record (EHR) review of a random sampling of patients ≥ 65 years with a historical diagnosis code of dementia who visited an ED within a large regional health network between 1/2014 and 1/2022. ED notes within the EHRs were coded for documentation of care partner communication and presence of a care partner in the ED. Logistic regression was used to identify patient characteristics associated with the composite outcome of either care partner communication or care partner presence in the ED. Results: A total of 460 patients were included. The median age was 83.0 years, 59.3% were female, 11.3% were Black, and 7.6% Hispanic. A care partner was documented in the ED for 22.4% of the visits and care partner communication documented for 43.9% of visits. 54.8% of patients had no documentation of care partner communication nor evidence of a care partner at the bedside. In multivariate logistic regression, increasing age (OR, (95% CI): 1.06 (1.04-1.09)), altered mental status (OR: 2.26 (1.01-5.05)), and weakness (OR: 3.38 (1.49-7.65)) significantly increased the probability of having care partner communication documented or a care partner at the bedside. Conclusion: More than half of PLwD in our sample did not have clinician documentation of communication with a care partner or a care partner in the ED. Further studies are needed to use these insights to improve communication with care partners of PLwD in the ED.

Analysis of Health Trajectories Leading to Adverse Opioid-Related Events.

Identifying patient risk factors leading to adverse opioid-related events (AOEs) may enable targeted risk-based interventions, uncover potential causal mechanisms, and enhance prognosis. In this article, we aim to discover patient diagnosis, procedure, and medication event trajectories associated with AOEs using large-scale data mining methods. The individual temporally preceding factors associated with the highest relative risk (RR) for AOEs were opioid withdrawal therapy agents, toxic encephalopathy, problems related to housing and economic circumstances, and unspecified viral hepatitis, with RR of 33.4, 26.1, 19.9, and 18.7, respectively. Patient cohorts with a socioeconomic or mental health code had a larger RR for over 75% of all identified trajectories compared to the average population. By analyzing health trajectories leading to AOEs, we discover novel, temporally-connected combinations of diagnoses and health service events that significantly increase risk of AOEs, including natural histories marked by socioeconomic and mental health diagnoses.