POINT: Pipeline for Offline Conversion and Integration of Geocodes and Neighborhood Data.

OBJECTIVES: Geocoding, the process of converting addresses into precise geographic coordinates, allows researchers and health systems to obtain neighborhood-level estimates of social determinants of health. This information supports opportunities to personalize care and interventions for individual patients based on the environments where they live. We developed an integrated offline geocoding pipeline to streamline the process of obtaining address-based variables, which can be integrated into existing data processing pipelines. METHODS: POINT is a web-based, containerized, application for geocoding addresses that can be deployed offline and made available to multiple users across an organization. Our application supports use through both a graphical user interface and application programming interface to query geographic variables, by census tract, without exposing sensitive patient data. We evaluated our application's performance using two datasets: one consisting of 1 million nationally representative addresses sampled from Open Addresses, and the other consisting of 3,096 previously geocoded patient addresses. RESULTS: A total of 99.4 and 99.8% of addresses in the Open Addresses and patient addresses datasets, respectively, were geocoded successfully. Census tract assignment was concordant with reference in greater than 90% of addresses for both datasets. Among successful geocodes, median (interquartile range) distances from reference coordinates were 52.5 (26.5-119.4) and 14.5 (10.9-24.6) m for the two datasets. CONCLUSION: POINT successfully geocodes more addresses and yields similar accuracy to existing solutions, including the U.S. Census Bureau's official geocoder. Addresses are considered protected health information and cannot be shared with common online geocoding services. POINT is an offline solution that enables scalability to multiple users and integrates downstream mapping to neighborhood-level variables with a pipeline that allows users to incorporate additional datasets as they become available. As health systems and researchers continue to explore and improve health equity, it is essential to quickly and accurately obtain neighborhood variables in a Health Insurance Portability and Accountability Act (HIPAA)-compliant way.

Assessment of a Naloxone Coprescribing Alert for Patients at Risk of Opioid Overdose: A Quality Improvement Project.

BACKGROUND: Patients taking high doses of opioids, or taking opioids in combination with other central nervous system depressants, are at increased risk of opioid overdose. Coprescribing the opioid-reversal agent naloxone is an essential safety measure, recommended by the surgeon general, but the rate of naloxone coprescribing is low. Therefore, we set out to determine whether a targeted clinical decision support alert could increase the rate of naloxone coprescribing. METHODS: We conducted a before-after study from January 2019 to April 2021 at a large academic health system in the Southeast. We developed a targeted point of care decision support notification in the electronic health record to suggest ordering naloxone for patients who have a high risk of opioid overdose based on a high morphine equivalent daily dose (MEDD) ≥90 mg, concomitant benzodiazepine prescription, or a history of opioid use disorder or opioid overdose. We measured the rate of outpatient naloxone prescribing as our primary measure. A multivariable logistic regression model with robust variance to adjust for prescriptions within the same prescriber was implemented to estimate the association between alerts and naloxone coprescribing. RESULTS: The baseline naloxone coprescribing rate in 2019 was 0.28 (95% confidence interval [CI], 0.24-0.31) naloxone prescriptions per 100 opioid prescriptions. After alert implementation, the naloxone coprescribing rate increased to 4.51 (95% CI, 4.33-4.68) naloxone prescriptions per 100 opioid prescriptions (P < .001). The adjusted odds of naloxone coprescribing after alert implementation were approximately 28 times those during the baseline period (95% CI, 15-52). CONCLUSIONS: A targeted decision support alert for patients at risk for opioid overdose significantly increased the rate of naloxone coprescribing and was relatively easy to build.

Terminations in Primary Care: a Retrospective Observational Study of 16 Primary Care Clinics.

BACKGROUND: The relationship between clinician and patient is the cornerstone of primary care. Breakdown and termination of this relationship are understudied yet important, undesirable outcomes. OBJECTIVE: To better understand the nature and extent of provider and clinic termination of the primary care relationship. DESIGN: Retrospective observational case-control study. SUBJECTS: Adult patients in Eastern Massachusetts who received primary care at hospital- and community-based clinics and health centers participating in a practice-based research network between January 2013 and June 2017. MAIN MEASURES: Formal termination by primary care physician (PCP), reasons for termination, independent predictors of termination based on mixed-effects logistic regression, and documentation of a new PCP after termination. KEY RESULTS: We identified 158,192 patients who received primary care from 182 PCPs across 16 clinics. We found 536 cases of formal termination. Clinics ranged from 4 to 119 terminations per 10,000 patients (intraclass correlation coefficient [ICC]=0.21; 95% CI: 0.18-0.24). Patient age, race/ethnicity, educational attainment, relationship status, employment status, and insurance type were independent predictors of termination (e.g., compared to patients employed full-time, patients unemployed due to disability were more likely to be terminated [adjusted OR:9.26; 95% CI: 6.74-12.74]). The most common cause for termination (38%) was appointment "no-shows" with some PCPs/clinics found to enforce a policy of dismissal following three no-shows. At the time of chart review, 201 patients (38%) had no documentation of a new PCP. Among patients who re-established care within the network, 134 (25%) had a primary care visit within 6 months of termination. CONCLUSIONS: Detailed chart review found that, unlike previous survey-based studies, dismissal was often for missed appointments based on enforcement of no-show policies. Many sociodemographic factors were associated with termination. Variability among clinics highlights the need for further research to better understand circumstances surrounding terminations, with the principal goals of improving patient-provider relationships and providing equitable care.

Predicting Self-Rated Health Across the Life Course: Health Equity Insights from Machine Learning Models.

BACKGROUND: Self-rated health is a strong predictor of mortality and morbidity. Machine learning techniques may provide insights into which of the multifaceted contributors to self-rated health are key drivers in diverse groups. OBJECTIVE: We used machine learning algorithms to predict self-rated health in diverse groups in the Behavioral Risk Factor Surveillance System (BRFSS), to understand how machine learning algorithms might be used explicitly to examine drivers of self-rated health in diverse populations. DESIGN: We applied three common machine learning algorithms to predict self-rated health in the 2017 BRFSS survey, stratified by age, race/ethnicity, and sex. We replicated our process in the 2016 BRFSS survey. PARTICIPANTS: We analyzed data from 449,492 adult participants of the 2017 BRFSS survey. MAIN MEASURES: We examined area under the curve (AUC) statistics to examine model fit within each group. We used traditional logistic regression to predict self-rated health associated with features identified by machine learning models. KEY RESULTS: Each algorithm, regularized logistic regression (AUC: 0.81), random forest (AUC: 0.80), and support vector machine (AUC: 0.81), provided good model fit in the BRFSS. Predictors of self-rated health were similar by sex and race/ethnicity but differed by age. Socioeconomic features were prominent predictors of self-rated health in mid-life age groups. Income [OR: 1.70 (95% CI: 1.62-1.80)], education [OR: 2.02 (95% CI: 1.89, 2.16)], physical activity [OR: 1.52 (95% CI: 1.46-1.58)], depression [OR: 0.66 (95% CI: 0.63-0.68)], difficulty concentrating [OR: 0.62 (95% CI: 0.58-0.66)], and hypertension [OR: 0.59 (95% CI: 0.57-0.61)] all predicted the odds of excellent or very good self-rated health. CONCLUSIONS: Our analysis of BRFSS data show social determinants of health are prominent predictors of self-rated health in mid-life. Our work may demonstrate promising practices for using machine learning to advance health equity.

Recommendations for the safe, effective use of adaptive CDS in the US healthcare system: an AMIA position paper.

The development and implementation of clinical decision support (CDS) that trains itself and adapts its algorithms based on new data-here referred to as Adaptive CDS-present unique challenges and considerations. Although Adaptive CDS represents an expected progression from earlier work, the activities needed to appropriately manage and support the establishment and evolution of Adaptive CDS require new, coordinated initiatives and oversight that do not currently exist. In this AMIA position paper, the authors describe current and emerging challenges to the safe use of Adaptive CDS and lay out recommendations for the effective management and monitoring of Adaptive CDS.

SUPPORT-AF IV: Supporting use of AC through provider prompting about oral anticoagulation therapy for AF clinical trial study protocol.

Background: Six million Americans suffer from atrial fibrillation (AF), a heart rhythm abnormality that significantly increases the risk of stroke. AF is responsible for 15% of ischemic strokes, which lead to permanent disability in 60% of cases and death in up to 20%. Anticoagulation (AC) is the mainstay for stroke prevention in patients with AF. Despite guidelines recommending AC for patients, up to half of eligible patients are not on AC. Clinical decision support tools in the electronic health record (EHR) can help bridge the disparity in AC prescription for patients with AF. Objective: To enhance and assess the effectiveness of our previous rule-based alert on AC initiation and persistence in a diverse patient population from UMass-Memorial Medical Center and University of Florida at Jacksonville. Methods/Results: Using the EHR, we will track AC initiation and persistence. We will interview both patients and providers to determine a measure of satisfaction with AC management. We will track digital crumbs to better understand the alert's mechanism of effect and further add enhancements. These enhancements will be used to refine the alert and aid in developing an implementation toolkit to facilitate use of the alert at other health systems. Conclusion: If the number of AC starts, the likelihood of persisting on AC, and the frequency alert use are found to be higher among intervention vs control providers, we believe such findings will confirm our hypothesis on the effectiveness of our alert.

Application Programming Interfaces in Health Care: Findings from a Current-State Sociotechnical Assessment.

OBJECTIVE: Interest in application programming interfaces (APIs) is increasing as key stakeholders look for technical solutions to interoperability challenges. We explored three thematic areas to assess the current state of API use for data access and exchange in health care: (1) API use cases and standards; (2) challenges and facilitators for read and write capabilities; and (3) outlook for development of write capabilities. METHODS: We employed four methods: (1) literature review; (2) expert interviews with 13 API stakeholders; (3) review of electronic health record (EHR) app galleries; and (4) a technical expert panel. We used an eight-dimension sociotechnical model to organize our findings. RESULTS: The API ecosystem is complicated and cuts across five of the eight sociotechnical model dimensions: (1) app marketplaces support a range of use cases, the majority of which target providers' needs, with far fewer supporting patient access to data; (2) current focus on read APIs with limited use of write APIs; (3) where standards are used, they are largely Fast Healthcare Interoperability Resources (FHIR); (4) FHIR-based APIs support exchange of electronic health information within the common clinical data set; and (5) validating external data and data sources for clinical decision making creates challenges to provider workflows. CONCLUSION: While the use of APIs in health care is increasing rapidly, it is still in the pilot stages. We identified five key issues with implications for the continued advancement of API use: (1) a robust normative FHIR standard; (2) expansion of the common clinical data set to other data elements; (3) enhanced support for write implementation; (4) data provenance rules; and (5) data governance rules. Thus, while APIs are being touted as a solution to interoperability challenges, they remain an emerging technology that is only one piece of a multipronged approach to data access and use.

Application Programming Interfaces (APIs) in Health Care: Findings from a Current-State Assessment.

Interest in application programming interfaces (APIs) as a means to increase health data access and exchange among patients, health care providers, and payers has become an important area for development. In an effort to better understand the various contexts in which APIs can be applied, we explored different use cases. While APIs and our collective understanding of the best ways to implement and use them continue to develop, in the coming years the use of proprietary and standards-based APIs could be key to the sustainability of applied clinical informatics research, as well as associated improvements in patient engagement, clinical decision making, efficiency, quality and safety of the healthcare delivery system.

Assessment of Employee Susceptibility to Phishing Attacks at US Health Care Institutions.

Importance: Cybersecurity is an increasingly important threat to health care delivery, and email phishing is a major attack vector against hospital employees. Objective: To describe the practice of phishing simulation and the extent to which health care employees are vulnerable to phishing simulations. Design, Setting, and Participants: Retrospective, multicenter quality improvement study of a convenience sample of 6 geographically dispersed US health care institutions that ran phishing simulations from August 1, 2011, through April 10, 2018. The specific institutions are anonymized herein for security and privacy concerns. Exposures: Simulated phishing emails received by employees at US health care institutions. Main Outcomes and Measures: Date of phishing campaign, campaign number, number of emails sent, number of emails clicked, and email content. Emails were classified into 3 categories (office related, personal, or information technology related). Results: The final study sample included 6 anonymized US health care institutions, 95 simulated phishing campaigns, and 2 971 945 emails, 422 062 of which were clicked (14.2%). The median institutional click rates for campaigns ranged from 7.4% (interquartile range [IQR], 5.8%-9.6%) to 30.7% (IQR, 25.2%-34.4%), with an overall median click rate of 16.7% (IQR, 8.3%-24.2%) across all campaigns and institutions. In the regression model, repeated phishing campaigns were associated with decreased odds of clicking on a subsequent phishing email (adjusted OR, 0.511; 95% CI, 0.382-0.685 for 6-10 campaigns; adjusted OR, 0.335; 95% CI, 0.282-0.398 for >10 campaigns). Conclusions and Relevance: Among a sample of US health care institutions that sent phishing simulations, almost 1 in 7 simulated emails sent were clicked on by employees. Increasing campaigns were associated with decreased odds of clicking on a phishing email, suggesting a potential benefit of phishing simulation and awareness. With cyberattacks increasing against US health care systems, these click rates represent a major cybersecurity risk for hospitals.

Changes in hospital bond ratings after the transition to a new electronic health record.

Objective: To assess the impact of electronic health record (EHR) implementation on hospital finances. Materials and Methods: We analyzed the impact of EHR implementation on bond ratings and net income from service to patients (NISP) at 32 hospitals that recently implemented a new EHR and a set of controls. Results: After implementing an EHR, 7 hospitals had a bond downgrade, 7 had a bond upgrade, and 18 had no changes. There was no difference in the likelihood of bond rating changes or in changes to NISP following EHR go-live when compared to control hospitals. Discussion: Most hospitals in our analysis saw no change in bond ratings following EHR go-live, with no significant differences observed between EHR implementation and control hospitals. There was also no apparent difference in NISP. Conclusions: Implementation of an EHR did not appear to have an impact on bond ratings at the hospitals in our analysis.

Nephrology co-management versus primary care solo management for early chronic kidney disease: a retrospective cross-sectional analysis.

BACKGROUND: Primary care physicians (PCPs) typically manage early chronic kidney disease (CKD), but recent guidelines recommend nephrology co-management for some patients with stage 3 CKD and all patients with stage 4 CKD. We sought to compare quality of care for co-managed patients to solo managed patients. METHODS: We conducted a retrospective cross-sectional analysis. Patients included in the study were adults who visited a PCP during 2009 with laboratory evidence of CKD in the preceding two years, defined as two estimated glomerular filtration rates (eGFR) between 15-59 mL/min/1.73 m(2) separated by 90 days. We assessed process measures (serum eGFR test, urine protein/albumin test, angiotensin converting enzyme inhibitor or angiotensin receptor blocker [ACE/ARB] prescription, and several tests monitoring for complications) and intermediate clinical outcomes (mean blood pressure and blood pressure control) and performed subgroup analyses by CKD stage. RESULTS: Of 3118 patients, 11 % were co-managed by a nephrologist. Co-management was associated with younger age (69 vs. 74 years), male gender (46 % vs. 34 %), minority race/ethnicity (black 32 % vs. 22 %; Hispanic 13 % vs. 8 %), hypertension (75 % vs. 66 %), diabetes (42 % vs. 26 %), and more PCP visits (5.0 vs. 3.9; p < 0.001 for all comparisons). After adjustment, co-management was associated with serum eGFR test (98 % vs. 94 %, p = <0.0001), urine protein/albumin test (82 % vs 36 %, p < 0.0001), and ACE/ARB prescription (77 % vs. 69 %, p = 0.03). Co-management was associated with monitoring for anemia and metabolic bone disease, but was not associated with lipid monitoring, differences in mean blood pressure (133/69 mmHg vs. 131/70 mmHg, p > 0.50) or blood pressure control. A subgroup analysis of Stage 4 CKD patients did not show a significant association between co-management and ACE/ARB prescription (80 % vs. 73 %, p = 0.26). CONCLUSION: For stage 3 and 4 CKD patients, nephrology co-management was associated with increased stage-appropriate monitoring and ACE/ARB prescribing, but not improved blood pressure control.

Problem list completeness in electronic health records: A multi-site study and assessment of success factors.

OBJECTIVE: To assess problem list completeness using an objective measure across a range of sites, and to identify success factors for problem list completeness. METHODS: We conducted a retrospective analysis of electronic health record data and interviews at ten healthcare organizations within the United States, United Kingdom, and Argentina who use a variety of electronic health record systems: four self-developed and six commercial. At each site, we assessed the proportion of patients who have diabetes recorded on their problem list out of all patients with a hemoglobin A1c elevation>=7.0%, which is diagnostic of diabetes. We then conducted interviews with informatics leaders at the four highest performing sites to determine factors associated with success. Finally, we surveyed all the sites about common practices implemented at the top performing sites to determine whether there was an association between problem list management practices and problem list completeness. RESULTS: Problem list completeness across the ten sites ranged from 60.2% to 99.4%, with a mean of 78.2%. Financial incentives, problem-oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture were identified as success factors at the four hospitals with problem list completeness at or near 90.0%. DISCUSSION: Incomplete problem lists represent a global data integrity problem that could compromise quality of care and put patients at risk. There was a wide range of problem list completeness across the healthcare facilities. Nevertheless, some facilities have achieved high levels of problem list completeness, and it is important to better understand the factors that contribute to success to improve patient safety. CONCLUSION: Problem list completeness varies substantially across healthcare facilities. In our review of EHR systems at ten healthcare facilities, we identified six success factors which may be useful for healthcare organizations seeking to improve the quality of their problem list documentation: financial incentives, problem oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture.

Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development.

OBJECTIVE: Clinical decision support (CDS) is essential for delivery of high-quality, cost-effective, and safe healthcare. The authors sought to evaluate the CDS capabilities across electronic health record (EHR) systems. METHODS: We evaluated the CDS implementation capabilities of 8 Office of the National Coordinator for Health Information Technology Authorized Certification Body (ONC-ACB)-certified EHRs. Within each EHR, the authors attempted to implement 3 user-defined rules that utilized the various data and logic elements expected of typical EHRs and that represented clinically important evidenced-based care. The rules were: 1) if a patient has amiodarone on his or her active medication list and does not have a thyroid-stimulating hormone (TSH) result recorded in the last 12 months, suggest ordering a TSH; 2) if a patient has a hemoglobin A1c result >7% and does not have diabetes on his or her problem list, suggest adding diabetes to the problem list; and 3) if a patient has coronary artery disease on his or her problem list and does not have aspirin on the active medication list, suggest ordering aspirin. RESULTS: Most evaluated EHRs lacked some CDS capabilities; 5 EHRs were able to implement all 3 rules, and the remaining 3 EHRs were unable to implement any of the rules. One of these did not allow users to customize CDS rules at all. The most frequently found shortcomings included the inability to use laboratory test results in rules, limit rules by time, use advanced Boolean logic, perform actions from the alert interface, and adequately test rules. CONCLUSION: Significant improvements in the EHR certification and implementation procedures are necessary.

Predicting Health Care Utilization After Behavioral Health Referral Using Natural Language Processing and Machine Learning.

Mental health problems are an independent predictor of increased healthcare utilization. We created random forest classifiers for predicting two outcomes following a patient's first behavioral health encounter: decreased utilization by any amount (AUROC 0.74) and ultra-high absolute utilization (AUROC 0.88). These models may be used for clinical decision support by referring providers, to automatically detect patients who may benefit from referral, for cost management, or for risk/protection factor analysis.

The use of sequential pattern mining to predict next prescribed medications.

BACKGROUND: Therapy for certain medical conditions occurs in a stepwise fashion, where one medication is recommended as initial therapy and other medications follow. Sequential pattern mining is a data mining technique used to identify patterns of ordered events. OBJECTIVE: To determine whether sequential pattern mining is effective for identifying temporal relationships between medications and accurately predicting the next medication likely to be prescribed for a patient. DESIGN: We obtained claims data from Blue Cross Blue Shield of Texas for patients prescribed at least one diabetes medication between 2008 and 2011, and divided these into a training set (90% of patients) and test set (10% of patients). We applied the CSPADE algorithm to mine sequential patterns of diabetes medication prescriptions both at the drug class and generic drug level and ranked them by the support statistic. We then evaluated the accuracy of predictions made for which diabetes medication a patient was likely to be prescribed next. RESULTS: We identified 161,497 patients who had been prescribed at least one diabetes medication. We were able to mine stepwise patterns of pharmacological therapy that were consistent with guidelines. Within three attempts, we were able to predict the medication prescribed for 90.0% of patients when making predictions by drug class, and for 64.1% when making predictions at the generic drug level. These results were stable under 10-fold cross validation, ranging from 89.1%-90.5% at the drug class level and 63.5-64.9% at the generic drug level. Using 1 or 2 items in the patient's medication history led to more accurate predictions than not using any history, but using the entire history was sometimes worse. CONCLUSION: Sequential pattern mining is an effective technique to identify temporal relationships between medications and can be used to predict next steps in a patient's medication regimen. Accurate predictions can be made without using the patient's entire medication history.

The Medicare Electronic Health Record Incentive Program: provider performance on core and menu measures.

OBJECTIVE: To measure performance by eligible health care providers on CMS's meaningful use measures. DATA SOURCE: Medicare Electronic Health Record Incentive Program Eligible Professionals Public Use File (PUF), which contains data on meaningful use attestations by 237,267 eligible providers through May 31, 2013. STUDY DESIGN: Cross-sectional analysis of the 15 core and 10 menu measures pertaining to use of EHR functions reported in the PUF. PRINCIPAL FINDINGS: Providers in the dataset performed strongly on all core measures, with the most frequent response for each of the 15 measures being 90-100 percent compliance, even when the threshold for a particular measure was lower (e.g., 30 percent). PCPs had higher scores than specialists for computerized order entry, maintaining an active medication list, and documenting vital signs, while specialists had higher scores for maintaining a problem list, recording patient demographics and smoking status, and for providing patients with an after-visit summary. In fact, 90.2 percent of eligible providers claimed at least one exclusion, and half claimed two or more. CONCLUSIONS: Providers are successfully attesting to CMS's requirements, and often exceeding the thresholds required by CMS; however, some troubling patterns in exclusions are present. CMS should raise program requirements in future years.

Use of a support vector machine for categorizing free-text notes: assessment of accuracy across two institutions.

BACKGROUND: Electronic health record (EHR) users must regularly review large amounts of data in order to make informed clinical decisions, and such review is time-consuming and often overwhelming. Technologies like automated summarization tools, EHR search engines and natural language processing have been shown to help clinicians manage this information. OBJECTIVE: To develop a support vector machine (SVM)-based system for identifying EHR progress notes pertaining to diabetes, and to validate it at two institutions. MATERIALS AND METHODS: We retrieved 2000 EHR progress notes from patients with diabetes at the Brigham and Women's Hospital (1000 for training and 1000 for testing) and another 1000 notes from the University of Texas Physicians (for validation). We manually annotated all notes and trained a SVM using a bag of words approach. We then used the SVM on the testing and validation sets and evaluated its performance with the area under the curve (AUC) and F statistics. RESULTS: The model accurately identified diabetes-related notes in both the Brigham and Women's Hospital testing set (AUC=0.956, F=0.934) and the external University of Texas Faculty Physicians validation set (AUC=0.947, F=0.935). DISCUSSION: Overall, the model we developed was quite accurate. Furthermore, it generalized, without loss of accuracy, to another institution with a different EHR and a distinct patient and provider population. CONCLUSIONS: It is possible to use a SVM-based classifier to identify EHR progress notes pertaining to diabetes, and the model generalizes well.

The future state of clinical data capture and documentation: a report from AMIA's 2011 Policy Meeting.

Much of what is currently documented in the electronic health record is in response toincreasingly complex and prescriptive medicolegal, reimbursement, and regulatory requirements. These requirements often result in redundant data capture and cumbersome documentation processes. AMIA's 2011 Health Policy Meeting examined key issues in this arena and envisioned changes to help move toward an ideal future state of clinical data capture and documentation. The consensus of the meeting was that, in the move to a technology-enabled healthcare environment, the main purpose of documentation should be to support patient care and improved outcomes for individuals and populations and that documentation for other purposes should be generated as a byproduct of care delivery. This paper summarizes meeting deliberations, and highlights policy recommendations and research priorities. The authors recommend development of a national strategy to review and amend public policies to better support technology-enabled data capture and documentation practices.