Mitigating Racial And Ethnic Bias And Advancing Health Equity In Clinical Algorithms: A Scoping Review.

In August 2022 the Department of Health and Human Services (HHS) issued a notice of proposed rulemaking prohibiting covered entities, which include health care providers and health plans, from discriminating against individuals when using clinical algorithms in decision making. However, HHS did not provide specific guidelines on how covered entities should prevent discrimination. We conducted a scoping review of literature published during the period 2011-22 to identify health care applications, frameworks, reviews and perspectives, and assessment tools that identify and mitigate bias in clinical algorithms, with a specific focus on racial and ethnic bias. Our scoping review encompassed 109 articles comprising 45 empirical health care applications that included tools tested in health care settings, 16 frameworks, and 48 reviews and perspectives. We identified a wide range of technical, operational, and systemwide bias mitigation strategies for clinical algorithms, but there was no consensus in the literature on a single best practice that covered entities could employ to meet the HHS requirements. Future research should identify optimal bias mitigation methods for various scenarios, depending on factors such as patient population, clinical setting, algorithm design, and types of bias to be addressed.

Machine learning functional impairment classification with electronic health record data.

BACKGROUND: Poor functional status is a key marker of morbidity, yet is not routinely captured in clinical encounters. We developed and evaluated the accuracy of a machine learning algorithm that leveraged electronic health record (EHR) data to provide a scalable process for identification of functional impairment. METHODS: We identified a cohort of patients with an electronically captured screening measure of functional status (Older Americans Resources and Services ADL/IADL) between 2018 and 2020 (N = 6484). Patients were classified using unsupervised learning K means and t-distributed Stochastic Neighbor Embedding into normal function (NF), mild to moderate functional impairment (MFI), and severe functional impairment (SFI) states. Using 11 EHR clinical variable domains (832 variable input features), we trained an Extreme Gradient Boosting supervised machine learning algorithm to distinguish functional status states, and measured prediction accuracies. Data were randomly split into training (80%) and test (20%) sets. The SHapley Additive Explanations (SHAP) feature importance analysis was used to list the EHR features in rank order of their contribution to the outcome. RESULTS: Median age was 75.3 years, 62% female, 60% White. Patients were classified as 53% NF (n = 3453), 30% MFI (n = 1947), and 17% SFI (n = 1084). Summary of model performance for identifying functional status state (NF, MFI, SFI) was AUROC (area under the receiving operating characteristic curve) 0.92, 0.89, and 0.87, respectively. Age, falls, hospitalization, home health use, labs (e.g., albumin), comorbidities (e.g., dementia, heart failure, chronic kidney disease, chronic pain), and social determinants of health (e.g., alcohol use) were highly ranked features in predicting functional status states. CONCLUSION: A machine learning algorithm run on EHR clinical data has potential utility for differentiating functional status in the clinical setting. Through further validation and refinement, such algorithms can complement traditional screening methods and result in a population-based strategy for identifying patients with poor functional status who need additional health resources.

Factors Associated With Health Care Professionals' Choice to Practice in Rural Minnesota.

Importance: Rural health inequities are due in part to a shortage of health care professionals in these areas. Objective: To determine the factors associated with health care professionals' decisions about where to practice. Design, Setting, and Participants: This prospective, cross-sectional survey study of health care professionals in Minnesota was administered by the Minnesota Department of Health from October 18, 2021, to July 25, 2022. Advanced practice registered nurses (APRNs), physicians, physician assistants (PAs), and registered nurses (RNs) renewing their professional licenses were eligible. Exposures: Individuals' ratings on survey items related to their choice of practice location. Main Outcomes and Measures: Rural or urban practice location as defined by the US Department of Agriculture's Rural-Urban Commuting Area typology. Results: A total of 32 086 respondents were included in the analysis (mean [SD] age, 44.4 [12.2] years; 22 728 identified as female [70.8%]). Response rates were 60.2% for APRNs (n = 2174), 97.7% for PAs (n = 2210), 95.1% for physicians (n = 11 019), and 61.6% for RNs (n = 16 663). The mean (SD) age of APRNs was 45.0 (10.3) years (1833 [84.3%] female); PAs, 39.0 (9.4) years (1648 [74.6%] female); physicians, 48.0 (11.9) years (4455 [40.4%] female); and RNs, 42.6 (12.3) years (14 792 [88.8%] female). Most respondents worked in urban (29 456 [91.8%]) vs rural (2630 [8.2%]) areas. Bivariate analysis suggested that family considerations are the most important determinant of practice location. Multivariate analysis revealed that having grown up in a rural area was the strongest factor associated with rural practice (odds ratio [OR] for APRNs, 3.44 [95% CI, 2.68-4.42]; OR for PAs, 3.75 [95% CI, 2.81-5.00]; OR for physicians, 2.44 [95% CI, 2.18-2.73]; OR for RNs, 3.77 [95% CI, 3.44-4.15]). When controlling for rural background, other associated factors included the availability of loan forgiveness (OR for APRNs, 1.42 [95% CI, 1.19-1.69]; OR for PAs, 1.60 [95% CI, 1.31-1.94]; OR for physicians, 1.54 [95% CI, 1.38-1.71]; OR for RNs, 1.20 [95% CI, 1.12-1.28]) and an educational program that prepared for rural practice (OR for APRNs, 1.44 [95% CI, 1.18-1.76]; OR for PAs. 1.70 [95% CI, 1.34-2.15]; OR for physicians, 1.31 [95% CI, 1.17-1.47]; OR for RNs, 1.23 [95% CI, 1.15-1.31]). Autonomy in one's work (OR for APRNs, 1.42 [95% CI, 1.08-1.86]; OR for PAs, 1.18 [95% CI, 0.89-1.58]; OR for physicians, 1.53 [95% CI, 1.31-1.78]; OR for RNs, 1.16 [95% CI, 1.07-1.25]) and a broad scope of practice (OR for APRNs, 1.46 [95% CI, 1.15-1.86]; OR for PAs, 0.96 [95% CI, 0.74-1.24]; OR for physicians, 1.62 [95% CI, 1.40-1.87]; OR for RNs, 0.96 [95% CI, 0.89-1.03]) were important factors associated with rural practice. Lifestyle and area considerations were not associated with rural practice; family considerations were associated with rural practice for RNs only (OR for APRNs, 0.97 [95% CI, 0.90-1.06]; OR for PAs, 0.95 [95% CI, 0.87-1.04]; OR for physicians, 0.92 [95% CI, 0.88-0.96]; OR for RNs, 1.05 [95% CI, 1.02-1.07]). Conclusions and Relevance: Understanding the interconnected factors involved in rural practice requires modeling relevant factors. The findings of this survey study suggest that loan forgiveness, rural training, autonomy, and a broad scope of practice are factors associated with rural practice for most health care professionals. Other factors associated with rural practice vary by profession, suggesting that there may not be a one-size-fits-all approach to recruitment of rural health care professionals.

Predictive Value of Early Autism Detection Models Based on Electronic Health Record Data Collected Before Age 1 Year.

Importance: Autism detection early in childhood is critical to ensure that autistic children and their families have access to early behavioral support. Early correlates of autism documented in electronic health records (EHRs) during routine care could allow passive, predictive model-based monitoring to improve the accuracy of early detection. Objective: To quantify the predictive value of early autism detection models based on EHR data collected before age 1 year. Design, Setting, and Participants: This retrospective diagnostic study used EHR data from children seen within the Duke University Health System before age 30 days between January 2006 and December 2020. These data were used to train and evaluate L2-regularized Cox proportional hazards models predicting later autism diagnosis based on data collected from birth up to the time of prediction (ages 30-360 days). Statistical analyses were performed between August 1, 2020, and April 1, 2022. Main Outcomes and Measures: Prediction performance was quantified in terms of sensitivity, specificity, and positive predictive value (PPV) at clinically relevant model operating thresholds. Results: Data from 45 080 children, including 924 (1.5%) meeting autism criteria, were included in this study. Model-based autism detection at age 30 days achieved 45.5% sensitivity and 23.0% PPV at 90.0% specificity. Detection by age 360 days achieved 59.8% sensitivity and 17.6% PPV at 81.5% specificity and 38.8% sensitivity and 31.0% PPV at 94.3% specificity. Conclusions and Relevance: In this diagnostic study of an autism screening test, EHR-based autism detection achieved clinically meaningful accuracy by age 30 days, improving by age 1 year. This automated approach could be integrated with caregiver surveys to improve the accuracy of early autism screening.

Serum ammonia use: unnecessary, frequent and costly.

Background/objective: While ammonia plays a role in the complex pathophysiology of hepatic encephalopathy (HE), serum ammonia is unreliable for both diagnosis of, and correlation with, neurological symptoms in patients with cirrhosis. We aimed to quantify ordering, cost and appropriate use of serum ammonia in a major Midwestern healthcare system. Design/method: Serum ammonia ordering in adult patients presenting to a large Midwestern health system was evaluated from 1 January 2015 to 31 December 2019. Results: Serum ammonia ordering was prevalent, with 20 338 tests ordered over 5 years. There were no differences in the number of inappropriate serum ammonia tests per 100 000 admissions for chronic liver disease over time (Pearson's correlation coefficient=-0.24, p=0.70). As a proportion of total ammonia tests ordered, inappropriate tests increased over time (Pearson's correlation coefficient=0.91, p=0.03). Inappropriate ordering was more common at community hospitals compared with the academic medical centre (99.3% vs 87.6%, p<0.001). Conclusion: Despite evidence that serum ammonia levels are unreliable for the diagnosis of HE and are not associated with severity of HE in individuals with cirrhosis, ordering remains prevalent, contributing to waste and potential harm.

Medical Student Workshop Improves Student Confidence in Working With Trained Medical Interpreters.

INTRODUCTION: Many physicians care for patients whose primary spoken language is not English, and these interactions present challenges in physician-patient communication. These challenges contribute to the significant health disparities experienced by populations with limited English proficiency (LEP). Using trained medical interpreters is an important step in addressing this problem, as it improves communication outcomes. Despite this, many medical education programs have little formal instruction on how to work effectively with interpreters. METHODS: To address this gap, we created an interactive workshop led by professional trained interpreters and faculty facilitators for medical students in their clinical years. Students were asked to evaluate the session based on relevance to their clinical experiences and helpfulness in preparing them for interactions with patients with LEP. RESULTS: Immediately after the session, students reported that the clinical scenarios presented were similar those seen on their clinical clerkships. They also reported increased confidence in their ability to work with interpreters. On later follow-up, students reported that the instruction helped prepare them for subsequent patient interactions that involved interpreters. CONCLUSION: A workshop is an effective method for improving medical student comfort and confidence when working with interpreters for populations with LEP.

Use of Hospital Referral Regions in Evaluating End-of-Life Care.

Background: Hospital referral regions (HRRs) are often used to characterize inpatient referral patterns, but it is unknown how well these geographic regions are aligned with variation in Medicare-financed hospice care, which is largely provided at home. Objective: Our objective was to characterize the variability in hospice use rates among elderly Medicare decedents by HRR and county. Methods: Using 2014 Master Beneficiary File for decedents 65 and older from North and South Carolina, we applied Bayesian mixed models to quantify variation in hospice use rates explained by HRR fixed effects, county random effects, and residual error among Medicare decedents. Results: We found HRRs and county indicators are significant predictors of hospice use in NC and SC; however, the relative variation within HRRs and associated residual variation is substantial. On average, HRR fixed effects explained more variation in hospice use rates than county indicators with a standard deviation (SD) of 10.0 versus 5.1 percentage points. The SD of the residual error is 5.7 percentage points. On average, variation within HRRs is about half the variation between regions (52%). Conclusions: The magnitude of unexplained residual variation in hospice use for NC and SC suggests that novel, end-of-life-specific service areas should be developed and tested to better capture geographic differences and inform research, health systems, and policy.

Paying for Palliative Care in Medicare: Evidence From the Four Seasons/Duke CMMI Demonstration.

CONTEXT: Palliative care improves patient and family outcomes and may reduce the cost of care, but this service is underutilized among Medicare beneficiaries. OBJECTIVES: To describe enrollment patterns and outcomes associated with the Center for Medicare and Medicaid Innovation expansion of a multisetting community palliative care program in North and South Carolina. METHODS: This observational study characterizes the Center for Medicare and Medicaid Innovation cohort's care and cost trajectories after enrollment. Program participants were age-eligible Medicare fee-for-service beneficiaries living in Western North Carolina and South Carolina who enrolled in a palliative care program from September 1, 2014, to August 31, 2017. End-of-life costs were compared between enrolled and nonenrolled decedents. Program administrative data and 100% Medicare claims data were used. RESULTS: A total of 5243 Medicare beneficiaries enrolled in the program from community (19%), facility (21%), small hospital (27%), or large hospital (33%) settings. Changes in Medicare expenditures in the 30 days after enrollment varied by setting. Adjusted odds of hospice use were 60% higher (OR = 1.60; CI = 1.47, 1.75) for enrolled decedents relative to nonenrolled decedents. Participants discharged to hospice vs. participants not had 17% (OR = 0.83 CI = 0.72, 0.94) lower costs. Among enrolled decedents those enrolled for at least 30 days vs. <30 days had 42% (OR = 0.58, CI = 0.49, 0.69) lower costs in the last 30 days of life. CONCLUSIONS: Expansion of community palliative care programs into multiple enrollment settings is feasible. It may improve hospice utilization among enrollees. Heterogeneous program participation by program setting pose challenges to a standardizing reimbursement policy.

The Assessment of Reasoning Tool (ART): structuring the conversation between teachers and learners.

Background Excellence in clinical reasoning is one of the most important outcomes of medical education programs, but assessing learners' reasoning to inform corrective feedback is challenging and unstandardized. Methods The Society to Improve Diagnosis in Medicine formed a multi-specialty team of medical educators to develop the Assessment of Reasoning Tool (ART). This paper describes the tool development process. The tool was designed to facilitate clinical teachers' assessment of learners' oral presentation for competence in clinical reasoning and facilitate formative feedback. Reasoning frameworks (e.g. script theory), contemporary practice goals (e.g. high-value care [HVC]) and proposed error reduction strategies (e.g. metacognition) were used to guide the development of the tool. Results The ART is a behaviorally anchored, three-point scale assessing five domains of reasoning: (1) hypothesis-directed data gathering, (2) articulation of a problem representation, (3) formulation of a prioritized differential diagnosis, (4) diagnostic testing aligned with HVC principles and (5) metacognition. Instructional videos were created for faculty development for each domain, guided by principles of multimedia learning. Conclusions The ART is a theory-informed assessment tool that allows teachers to assess clinical reasoning and structure feedback conversations.

Entrusting internal medicine residents to use point of care ultrasound: Towards improved assessment and supervision.

Background: Internal medicine physicians and trainees are increasingly using, and seeking training in, diagnostic point of care ultrasound (POCUS). Numerous internal medicine training programs have described their curricula, but little has been written about how learners should be assessed, supervised, and allowed to progress toward independent practice, yet these practices are imperative for safe and effective use. Entrustable professional activities (EPAs) offer a practical method to assess observable units of professional work and make supervision decisions. Methods: An EPA for POCUS is used as a framework to assess and determine appropriate levels of supervision in an internal medicine residency program. Results: All learners have been able to advance to level 2 with a mandatory introductory boot camp course. Learners have been able to advance to higher levels of independence, often after taking formal elective programmatic coursework. However, not all learners taking the same coursework have been granted the same level of independence. Conclusions: It is feasible to assess and supervise internal medicine residents' ability to use diagnostic point of care ultrasound using an EPA.

Diagnostic Discordance, Health Information Exchange, and Inter-Hospital Transfer Outcomes: a Population Study.

BACKGROUND: Studying diagnostic error at the population level requires an understanding of how diagnoses change over time. OBJECTIVE: To use inter-hospital transfers to examine the frequency and impact of changes in diagnosis on patient risk, and whether health information exchange can improve patient safety by enhancing diagnostic accuracy. DESIGN: Diagnosis coding before and after hospital transfer was merged with responses from the American Hospital Association Annual Survey for a cohort of patients transferred between hospitals to identify predictors of mortality. PARTICIPANTS: Patients (180,337) 18 years or older transferred between 473 acute care hospitals from NY, FL, IA, UT, and VT from 2011 to 2013. MAIN MEASURES: We identified discordant Elixhauser comorbidities before and after transfer to determine the frequency and developed a weighted score of diagnostic discordance to predict mortality. This was included in a multivariate model with inpatient mortality as the dependent variable. We investigated whether health information exchange (HIE) functionality adoption as reported by hospitals improved diagnostic discordance and inpatient mortality. KEY RESULTS: Discordance in diagnoses occurred in 85.5% of all patients. Seventy-three percent of patients gained a new diagnosis following transfer while 47% of patients lost a diagnosis. Diagnostic discordance was associated with increased adjusted inpatient mortality (OR 1.11 95% CI 1.10-1.11, p < 0.001) and allowed for improved mortality prediction. Bilateral hospital HIE participation was associated with reduced diagnostic discordance index (3.69 vs. 1.87%, p < 0.001) and decreased inpatient mortality (OR 0.88, 95% CI 0.89-0.99, p < 0.001). CONCLUSIONS: Diagnostic discordance commonly occurred during inter-hospital transfers and was associated with increased inpatient mortality. Health information exchange adoption was associated with decreased discordance and improved patient outcomes.

Will Changes to Medicare Payment Rates Alter Hospice's Cost-Saving Ability?

BACKGROUND: On January 1, 2016, Medicare implemented a new "two-tiered" model for hospice services, with per diem rates increased for days 1 through 60, decreased for days 61 and greater, and service intensity add-on payments made retrospectively for the last seven days of life. OBJECTIVE: To estimate whether the Medicare hospice benefit's potential for cost savings will change as a result of the January 2016 change in payment structure. DESIGN: Analysis of decedents' claims records using propensity score matching, logistic regression, and sensitivity analysis. SETTING/SUBJECTS: All age-eligible Medicare decedents who received care and died in North Carolina in calendar years 2009 and 2010. MEASUREMENTS: Costs to Medicare for hospice and other healthcare services. RESULTS: Medicare costs were reduced from hospice election until death using both 2009-2010 and new 2016 payment structures and rates. Mean cost savings were $1,527 with actual payment rates, and would have been $2,105 with the new payment rates (p < 0.001). Cost savings were confirmed by reducing the number of days used for cost comparison by three days for those with hospice stays of at least four days ($4,318 using 2009-2010 rates, $3,138 for 2016 rates: p < 0.001). Cost savings were greater for males ($3,393) versus females ($1,051) and greatest in cancer ($6,706) followed by debility and failure to thrive ($5,636) and congestive heart failure ($1,309); dementia patients had higher costs (+$1,880) (p < 0.001). When adding 3 days to the comparison period, hospice increased costs to Medicare. CONCLUSIONS: Medicare savings could continue with the 2016 payment rate change. Cost savings were found for all primary diagnoses analyzed except dementia.

Criterion-Based Assessment in a Norm-Based World: How Can We Move Past Grades?

In the United States, the medical education community has begun a shift from the Flexnerian time-based model to a competency-based medical education model. The graduate medical education (GME) community is substantially farther along in this transition than is the undergraduate medical education (UME) community.GME has largely adopted the use of competencies and their attendant milestones and increasingly is employing the framework of entrustable professional activities (EPAs) to assess trainee competence. The UME community faces several challenges to successfully navigating a similar transition. First is the reliance on norm-based reference standards in the UME-GME transition, comparing students' performance versus their peers' with grades, United States Medical Licensing Examination Step 1 and Step 2 score interpretation, and the structured Medical School Performance Evaluation, or dean's letter. Second is the reliance on proxy assessments rather than direct observation of learners. Third is the emphasis on summative rather than formative assessments.Educators have overcome a major barrier to change by establishing UME outcomes assessment criteria with the advent and general acceptance of the physician competency reference set and the Core EPAs for Entering Residency in UME. Now is the time for the hard work of developing assessments steeped in direct observation that can be accepted by learners and faculty across the educational continuum and can be shown to predict clinical performance in a much more meaningful way than the current measures of grades and examinations. The acceptance of such assessments will facilitate the UME transition toward competency-based medical education.

Antimicrobial stewardship in outpatient settings: a systematic review.

OBJECTIVE: Evaluate the effect of outpatient antimicrobial stewardship programs on prescribing, patient, microbial outcomes, and costs. DESIGN: Systematic review METHODS: Search of MEDLINE (2000 through November 2013), Cochrane Library, and reference lists of relevant studies. We included English language studies with patient populations relevant to the United States (eg, infectious conditions, prescription services) evaluating stewardship programs in outpatient settings and reporting outcomes of interest. Data regarding study characteristics and outcomes were extracted and organized by intervention type. RESULTS: We identified 50 studies eligible for inclusion, with most (29 of 50; 58%) reporting on respiratory tract infections, followed by multiple/unspecified infections (17 of 50; 34%). We found medium-strength evidence that stewardship programs incorporating communication skills training and laboratory testing are associated with reductions in antimicrobial use, and low-strength evidence that other stewardship interventions are associated with improved prescribing. Patient-centered outcomes, which were infrequently reported, were not adversely affected. Medication costs were generally lower with stewardship interventions, but overall program costs were rarely reported. No studies reported microbial outcomes, and data regarding outpatient settings other than primary care clinics are limited. CONCLUSIONS: Low- to moderate-strength evidence suggests that antimicrobial stewardship programs in outpatient settings improve antimicrobial prescribing without adversely effecting patient outcomes. Effectiveness depends on program type. Most studies were not designed to measure patient or resistance outcomes. Data regarding sustainability and scalability of interventions are limited.