Clinical, social, and policy factors in COVID-19 cases and deaths: methodological considerations for feature selection and modeling in county-level analyses.

BACKGROUND: There is a need to evaluate how the choice of time interval contributes to the lack of consistency of SDoH variables that appear as important to COVID-19 disease burden within an analysis for both case counts and death counts. METHODS: This study identified SDoH variables associated with U.S county-level COVID-19 cumulative case and death incidence for six different periods: the first 30, 60, 90, 120, 150, and 180 days since each county had COVID-19 one case per 10,000 residents. The set of SDoH variables were in the following domains: resource deprivation, access to care/health resources, population characteristics, traveling behavior, vulnerable populations, and health status. A generalized variance inflation factor (GVIF) analysis was used to identify variables with high multicollinearity. For each dependent variable, a separate model was built for each of the time periods. We used a mixed-effect generalized linear modeling of counts normalized per 100,000 population using negative binomial regression. We performed a Kolmogorov-Smirnov goodness of fit test, an outlier test, and a dispersion test for each model. Sensitivity analysis included altering the county start date to the day each county reached 10 COVID-19 cases per 10,000. RESULTS: Ninety-seven percent (3059/3140) of the counties were represented in the final analysis. Six features proved important for both the main and sensitivity analysis: adults-with-college-degree, days-sheltering-in-place-at-start, prior-seven-day-median-time-home, percent-black, percent-foreign-born, over-65-years-of-age, black-white-segregation, and days-since-pandemic-start. These variables belonged to the following categories: COVID-19 related, vulnerable populations, and population characteristics. Our diagnostic results show that across our outcomes, the models of the shorter time periods (30 days, 60 days, and 900 days) have a better fit. CONCLUSION: Our findings demonstrate that the set of SDoH features that are significant for COVID-19 outcomes varies based on the time from the start date of the pandemic and when COVID-19 was present in a county. These results could assist researchers with variable selection and inform decision makers when creating public health policy.

Rapid Evidence Review of Mobile Applications for Self-management of Diabetes.

BACKGROUND: Patients with diabetes lack information on which commercially available applications (apps) improve diabetes-related outcomes. We conducted a rapid evidence review to examine features, clinical efficacy, and usability of apps for self-management of type 1 and type 2 diabetes in adults. METHODS: Ovid/Medline and the Cochrane Database of Systematic Reviews were searched for systematic reviews and technology assessments. Reference lists of relevant systematic reviews were examined for primary studies. Additional searches for primary studies were conducted online, through Ovid/Medline, Embase, CINAHL, and ClinicalTrials.gov . Studies were evaluated for eligibility based on predetermined criteria, data were extracted, study quality was assessed using a risk of bias tool, information on app features was collected, and app usability was assessed. Results are summarized qualitatively. RESULTS: Fifteen articles evaluating 11 apps were identified: six apps for type 1 and five apps for type 2 diabetes. Common features of apps included setting reminders and tracking blood glucose and hemoglobin A1c (HbA1c), medication use, physical activity, and weight. Compared with controls, use of eight apps, when paired with support from a healthcare provider or study staff, improved at least one outcome, most often HbA1c. Patients did not experience improvements in quality of life, blood pressure, or weight, regardless of app used or type of diabetes. Study quality was variable. Of the eight apps available for usability testing, two were scored "acceptable," three were "marginal," and three were "not acceptable." DISCUSSION: Limited evidence suggests that use of some commercially available apps, when combined with additional support from a healthcare provider or study staff, may improve some short-term diabetes-related outcomes. The impact of these apps on longer-term outcomes is unclear. More rigorous and longer-term studies of apps are needed. REGISTRATION: This review was funded by the Agency for Healthcare Research and Quality (AHRQ). The protocol is available at: http://www.effectivehealthcare.ahrq.gov/topics/diabetes-mobile-devices/research-protocol .

Medical Record Closure Practices of Physicians Before and After the Use of Medical Scribes.

This study uses electronic health record data to evaluate medical record closure outcomes before and after the use of medical scribes at a large academic medical center.

Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research.

BACKGROUND: To demonstrate that subject selection based on sufficient laboratory results and medication orders in electronic health records can be biased towards sick patients. METHODS: Using electronic health record data from 10,000 patients who received anesthetic services at a major metropolitan tertiary care academic medical center, an affiliated hospital for women and children, and an affiliated urban primary care hospital, the correlation between patient health status and counts of days with laboratory results or medication orders, as indicated by the American Society of Anesthesiologists Physical Status Classification (ASA Class), was assessed with a Negative Binomial Regression model. RESULTS: Higher ASA Class was associated with more points of data: compared to ASA Class 1 patients, ASA Class 4 patients had 5.05 times the number of days with laboratory results and 6.85 times the number of days with medication orders, controlling for age, sex, emergency status, admission type, primary diagnosis, and procedure. CONCLUSIONS: Imposing data sufficiency requirements for subject selection allows researchers to minimize missing data when reusing electronic health records for research, but introduces a bias towards the selection of sicker patients. We demonstrated the relationship between patient health and quantity of data, which may result in a systematic bias towards the selection of sicker patients for research studies and limit the external validity of research conducted using electronic health record data. Additionally, we discovered other variables (i.e., admission status, age, emergency classification, procedure, and diagnosis) that independently affect data sufficiency.

Prediction of multiclass surgical outcomes in glaucoma using multimodal deep learning based on free-text operative notes and structured EHR data.

OBJECTIVE: Surgical outcome prediction is challenging but necessary for postoperative management. Current machine learning models utilize pre- and post-op data, excluding intraoperative information in surgical notes. Current models also usually predict binary outcomes even when surgeries have multiple outcomes that require different postoperative management. This study addresses these gaps by incorporating intraoperative information into multimodal models for multiclass glaucoma surgery outcome prediction. MATERIALS AND METHODS: We developed and evaluated multimodal deep learning models for multiclass glaucoma trabeculectomy surgery outcomes using both structured EHR data and free-text operative notes. We compare those to baseline models that use structured EHR data exclusively, or neural network models that leverage only operative notes. RESULTS: The multimodal neural network had the highest performance with a macro AUROC of 0.750 and F1 score of 0.583. It outperformed the baseline machine learning model with structured EHR data alone (macro AUROC of 0.712 and F1 score of 0.486). Additionally, the multimodal model achieved the highest recall (0.692) for hypotony surgical failure, while the surgical success group had the highest precision (0.884) and F1 score (0.775). DISCUSSION: This study shows that operative notes are an important source of predictive information. The multimodal predictive model combining perioperative notes and structured pre- and post-op EHR data outperformed other models. Multiclass surgical outcome prediction can provide valuable insights for clinical decision-making. CONCLUSIONS: Our results show the potential of deep learning models to enhance clinical decision-making for postoperative management. They can be applied to other specialties to improve surgical outcome predictions.

Variable Impact of Medical Scribes on Physician Electronic Health Record Documentation Practices: A Quantitative Analysis Across a Large, Integrated Health-System.

BACKGROUND: Medical scribes have been utilized to reduce electronic health record (EHR) associated documentation burden. Although evidence suggests benefits to scribes, no large-scale studies have quantitatively evaluated scribe impact on physician documentation across clinical settings. This study aimed to evaluate the effect of scribes on physician EHR documentation behaviors and performance. METHODS: This retrospective cohort study used EHR audit log data from a large academic health system to evaluate clinical documentation for all ambulatory encounters between January 2014 and December 2019 to evaluate the effect of scribes on physician documentation behaviors. Scribe services were provided on a first-come, first-served basis on physician request. Based on a physician's scribe use, encounters were grouped into 3 categories: never using a scribe, prescribe (before scribe use), or using a scribe. Outcomes included chart closure time, the proportion of delinquent charts, and charts closed after-hours. RESULTS: Three hundred ninety-five physicians (23% scribe users) across 29 medical subspecialties, encompassing 1,132,487 encounters, were included in the analysis. At baseline, scribe users had higher chart closure time, delinquent charts, and after-hours documentation than physicians who never used scribes. Among scribe users, the difference in outcome measures postscribe compared with baseline varied, and using a scribe rarely resulted in outcome measures approaching a range similar to the performance levels of nonusing physicians. In addition, there was variability in outcome measures across medical specialties and within similar subspecialties. CONCLUSION: Although scribes may improve documentation efficiency among some physicians, not all will improve EHR-related documentation practices. Different strategies may help to optimize documentation behaviors of physician-scribe dyads and maximize outcomes of scribe implementation.

Racial Disparities in Diabetes Care and Outcomes for Patients with Visual Impairment: A Descriptive Analysis of the TriNetX Research Network.

Background: This research delves into the confluence of racial disparities and health inequities among individuals with disabilities, with a focus on those contending with both diabetes and visual impairment. Methods: Utilizing data from the TriNetX Research Network, which includes electronic medical records of roughly 115 million patients from 83 anonymous healthcare organizations, this study employs a directed acyclic graph (DAG) to pinpoint confounders and augment interpretation. We identified patients with visual impairments using ICD-10 codes, deliberately excluding diabetes-related ophthalmology complications. Our approach involved multiple race-stratified analyses, comparing co-morbidities like chronic pulmonary disease in visually impaired patients against their counterparts. We assessed healthcare access disparities by examining the frequency of annual visits, instances of two or more A1c measurements, and glomerular filtration rate (GFR) measurements. Additionally, we evaluated diabetes outcomes by comparing the risk ratio of uncontrolled diabetes (A1c > 9.0) and chronic kidney disease in patients with and without visual impairments. Results: The incidence of diabetes was substantially higher (nearly double) in individuals with visual impairments across White, Asian, and African American populations. Higher rates of chronic kidney disease were observed in visually impaired individuals, with a risk ratio of 1.79 for African American, 2.27 for White, and non-significant for the Asian group. A statistically significant difference in the risk ratio for uncontrolled diabetes was found only in the White cohort (0.843). White individuals without visual impairments were more likely to receive two A1c tests, a trend not significant in other racial groups. African Americans with visual impairments had a higher rate of glomerular filtration rate testing. However, White individuals with visual impairments were less likely to undergo GFR testing, indicating a disparity in kidney health monitoring. This pattern of disparity was not observed in the Asian cohort. Conclusions: This study uncovers pronounced disparities in diabetes incidence and management among individuals with visual impairments, particularly among White, Asian, and African American groups. Our DAG analysis illuminates the intricate interplay between SDoH, healthcare access, and frequency of crucial diabetes monitoring practices, highlighting visual impairment as both a medical and social issue.

Defining and measuring completeness of electronic health records for secondary use.

We demonstrate the importance of explicit definitions of electronic health record (EHR) data completeness and how different conceptualizations of completeness may impact findings from EHR-derived datasets. This study has important repercussions for researchers and clinicians engaged in the secondary use of EHR data. We describe four prototypical definitions of EHR completeness: documentation, breadth, density, and predictive completeness. Each definition dictates a different approach to the measurement of completeness. These measures were applied to representative data from NewYork-Presbyterian Hospital's clinical data warehouse. We found that according to any definition, the number of complete records in our clinical database is far lower than the nominal total. The proportion that meets criteria for completeness is heavily dependent on the definition of completeness used, and the different definitions generate different subsets of records. We conclude that the concept of completeness in EHR is contextual. We urge data consumers to be explicit in how they define a complete record and transparent about the limitations of their data.

Healthcare utilization is a collider: an introduction to collider bias in EHR data reuse.

OBJECTIVES: Collider bias is a common threat to internal validity in clinical research but is rarely mentioned in informatics education or literature. Conditioning on a collider, which is a variable that is the shared causal descendant of an exposure and outcome, may result in spurious associations between the exposure and outcome. Our objective is to introduce readers to collider bias and its corollaries in the retrospective analysis of electronic health record (EHR) data. TARGET AUDIENCE: Collider bias is likely to arise in the reuse of EHR data, due to data-generating mechanisms and the nature of healthcare access and utilization in the United States. Therefore, this tutorial is aimed at informaticians and other EHR data consumers without a background in epidemiological methods or causal inference. SCOPE: We focus specifically on problems that may arise from conditioning on forms of healthcare utilization, a common collider that is an implicit selection criterion when one reuses EHR data. Directed acyclic graphs (DAGs) are introduced as a tool for identifying potential sources of bias during study design and planning. References for additional resources on causal inference and DAG construction are provided.

Electronic health record data quality assessment and tools: a systematic review.

OBJECTIVE: We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies. MATERIALS AND METHODS: We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process. RESULTS: We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology. DISCUSSION: There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality. CONCLUSION: Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

The Role of Informatics in Implementing Guidelines for Chronic Opioid Therapy Risk Assessment in Primary Care: A Narrative Review Informed by the Socio-Technical Model.

Approximately 2 million Americans live with opioid use disorder (OUD), most of whom also have chronic pain. The economic burden of chronic pain and prescription opioid misuse runs into billions of dollars. Patients on prescription opioids for chronic non-cancer pain (CNCP) are at increased risk for OUD and overdose. By adhering to the Center for Disease Control and Prevention (CDC) opioid prescribing guidelines, primary care providers (PCPs) have the potential to improve patient outcomes. But numerous provider, patient, and practice-specific factors challenge adherence to guidelines in primary care. Many of the barriers may be mediated by informatics interventions, but gaps in knowledge and unmet needs exist. This narrative review examines the risk assessment and harm reduction process in a socio-technical context to highlight the gaps in knowledge and unmet needs that can be mediated through informatics intervention.

Issues With Variability in Electronic Health Record Data About Race and Ethnicity: Descriptive Analysis of the National COVID Cohort Collaborative Data Enclave.

BACKGROUND: The adverse impact of COVID-19 on marginalized and under-resourced communities of color has highlighted the need for accurate, comprehensive race and ethnicity data. However, a significant technical challenge related to integrating race and ethnicity data in large, consolidated databases is the lack of consistency in how data about race and ethnicity are collected and structured by health care organizations. OBJECTIVE: This study aims to evaluate and describe variations in how health care systems collect and report information about the race and ethnicity of their patients and to assess how well these data are integrated when aggregated into a large clinical database. METHODS: At the time of our analysis, the National COVID Cohort Collaborative (N3C) Data Enclave contained records from 6.5 million patients contributed by 56 health care institutions. We quantified the variability in the harmonized race and ethnicity data in the N3C Data Enclave by analyzing the conformance to health care standards for such data. We conducted a descriptive analysis by comparing the harmonized data available for research purposes in the database to the original source data contributed by health care institutions. To make the comparison, we tabulated the original source codes, enumerating how many patients had been reported with each encoded value and how many distinct ways each category was reported. The nonconforming data were also cross tabulated by 3 factors: patient ethnicity, the number of data partners using each code, and which data models utilized those particular encodings. For the nonconforming data, we used an inductive approach to sort the source encodings into categories. For example, values such as "Declined" were grouped with "Refused," and "Multiple Race" was grouped with "Two or more races" and "Multiracial." RESULTS: "No matching concept" was the second largest harmonized concept used by the N3C to describe the race of patients in their database. In addition, 20.7% of the race data did not conform to the standard; the largest category was data that were missing. Hispanic or Latino patients were overrepresented in the nonconforming racial data, and data from American Indian or Alaska Native patients were obscured. Although only a small proportion of the source data had not been mapped to the correct concepts (0.6%), Black or African American and Hispanic/Latino patients were overrepresented in this category. CONCLUSIONS: Differences in how race and ethnicity data are conceptualized and encoded by health care institutions can affect the quality of the data in aggregated clinical databases. The impact of data quality issues in the N3C Data Enclave was not equal across all races and ethnicities, which has the potential to introduce bias in analyses and conclusions drawn from these data. Transparency about how data have been transformed can help users make accurate analyses and inferences and eventually better guide clinical care and public policy.

Social Determinants of Health Factors for Gene-Environment COVID-19 Research: Challenges and Opportunities.

The characteristics of a person's health status are often guided by how they live, grow, learn, their genetics, as well as their access to health care. Yet, all too often, studies examining the relationship between social determinants of health (behavioral, sociocultural, and physical environmental factors), the role of demographics, and health outcomes poorly represent these relationships, leading to misinterpretations, limited study reproducibility, and datasets with limited representativeness and secondary research use capacity. This is a profound hurdle in what questions can or cannot be rigorously studied about COVID-19. In practice, gene-environment interactions studies have paved the way for including these factors into research. Similarly, our understanding of social determinants of health continues to expand with diverse data collection modalities as health systems, patients, and community health engagement aim to fill the knowledge gaps toward promoting health and wellness. Here, a conceptual framework is proposed, adapted from the population health framework, socioecological model, and causal modeling in gene-environment interaction studies to integrate the core constructs from each domain with practical considerations needed for multidisciplinary science.

Comparing ascertainment of chronic condition status with problem lists versus encounter diagnoses from electronic health records.

OBJECTIVE: To assess and compare electronic health record (EHR) documentation of chronic disease in problem lists and encounter diagnosis records among Community Health Center (CHC) patients. MATERIALS AND METHODS: We assessed patient EHR data in a large clinical research network during 2012-2019. We included CHCs who provided outpatient, older adult primary care to patients age ≥45 years, with ≥2 office visits during the study. Our study sample included 1 180 290 patients from 545 CHCs across 22 states. We used diagnosis codes from 39 Chronic Condition Warehouse algorithms to identify chronic conditions from encounter diagnoses only and compared against problem list records. We measured correspondence including agreement, kappa, prevalence index, bias index, and prevalence-adjusted bias-adjusted kappa. RESULTS: Overlap of encounter diagnosis and problem list ascertainment was 59.4% among chronic conditions identified, with 12.2% of conditions identified only in encounters and 28.4% identified only in problem lists. Rates of coidentification varied by condition from 7.1% to 84.4%. Greatest agreement was found in diabetes (84.4%), HIV (78.1%), and hypertension (74.7%). Sixteen conditions had <50% agreement, including cancers and substance use disorders. Overlap for mental health conditions ranged from 47.4% for anxiety to 59.8% for depression. DISCUSSION: Agreement between the 2 sources varied substantially. Conditions requiring regular management in primary care settings may have a higher agreement than those diagnosed and treated in specialty care. CONCLUSION: Relying on EHR encounter data to identify chronic conditions without reference to patient problem lists may under-capture conditions among CHC patients in the United States.

Unraveling COVID-19: A Large-Scale Characterization of 4.5 Million COVID-19 Cases Using CHARYBDIS.

Purpose: Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) Characterizing Health Associated Risks and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD. Patients and Methods: We conducted a descriptive retrospective database study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11th June 2020 and are iteratively updated via GitHub. We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19, and 113,627 hospitalized with COVID-19 requiring intensive services. Results: We aggregated over 22,000 unique characteristics describing patients with COVID-19. All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts and are readily available online. Globally, we observed similarities in the USA and Europe: more women diagnosed than men but more men hospitalized than women, most diagnosed cases between 25 and 60 years of age versus most hospitalized cases between 60 and 80 years of age. South Korea differed with more women than men hospitalized. Common comorbidities included type 2 diabetes, hypertension, chronic kidney disease and heart disease. Common presenting symptoms were dyspnea, cough and fever. Symptom data availability was more common in hospitalized cohorts than diagnosed. Conclusion: We constructed a global, multi-centre view to describe trends in COVID-19 progression, management and evolution over time. By characterising baseline variability in patients and geography, our work provides critical context that may otherwise be misconstrued as data quality issues. This is important as we perform studies on adverse events of special interest in COVID-19 vaccine surveillance.

The quality of social determinants data in the electronic health record: a systematic review.

OBJECTIVE: The aim of this study was to collect and synthesize evidence regarding data quality problems encountered when working with variables related to social determinants of health (SDoH). MATERIALS AND METHODS: We conducted a systematic review of the literature on social determinants research and data quality and then iteratively identified themes in the literature using a content analysis process. RESULTS: The most commonly represented quality issue associated with SDoH data is plausibility (n = 31, 41%). Factors related to race and ethnicity have the largest body of literature (n = 40, 53%). The first theme, noted in 62% (n = 47) of articles, is that bias or validity issues often result from data quality problems. The most frequently identified validity issue is misclassification bias (n = 23, 30%). The second theme is that many of the articles suggest methods for mitigating the issues resulting from poor social determinants data quality. We grouped these into 5 suggestions: avoid complete case analysis, impute data, rely on multiple sources, use validated software tools, and select addresses thoughtfully. DISCUSSION: The type of data quality problem varies depending on the variable, and each problem is associated with particular forms of analytical error. Problems encountered with the quality of SDoH data are rarely distributed randomly. Data from Hispanic patients are more prone to issues with plausibility and misclassification than data from other racial/ethnic groups. CONCLUSION: Consideration of data quality and evidence-based quality improvement methods may help prevent bias and improve the validity of research conducted with SDoH data.

An Analysis of Two Sources of Cardiology Patient Data to Measure Medication Agreement.

Errors and incompleteness in electronic health record (EHR) medication lists can result in medical errors. To reduce errors in these medication lists, clinicians use patient self-reported data to reconcile EHR data. We assessed the agreement between patient self-reported medications and medications recorded in the EHR for six medication classes related to cardiovascular care and used logistic regression models to determine which patient-related factors were associated with the disagreement between these two information sources. From our 297 patients, we found self-reported medications had an overall above-average agreement with the EHR (? = .727). We observed the highest agreement level for statins (? = .831) and the lowest for other antihypertensives (? = .465). Agreement was less likely for Hispanic and male patients. We also performed an in-depth error analysis of different types of disagreement beyond medication names, which revealed that the most frequent type of disagreement was mismatched dosages.

Extracting Patient-level Social Determinants of Health into the OMOP Common Data Model.

Deficiencies in data sharing capabilities limit Social Determinants of Health (SDoH) analysis as part of COVID-19 research. The National COVID Cohort Collaborative (N3C) is an example of an Electronic Health Record (EHR) database of patients tested for COVID-19 that could benefit from a SDoH elements framework that captures various screening instruments in EHR data warehouse systems. This paper uses the University of Washington Enterprise Data Warehouse (a data contributor to N3C) to demonstrate how SDoH can be represented and managed to be made available within an OMOP common data model. We found that these data varied by type of social determinants data and where it was collected, in the time period that it was collected, and in how it was represented.

Chart Completion Time of Attending Physicians While Using Medical Scribes.

Medical scribes have become a widely used strategy to optimize how providers document in the electronic health record. To date, literature regarding the impact of scribes on time to complete documentation is limited. We conducted a retrospective, descriptive study of chart completion time among providers using scribes at our organization. A total of 148,410 scribed encounters, across 55 different clinics, were analyzed to determine variations in chart completion time. There was a significant variance in completion time between specialty groups and clinics within each specialty. Additionally, chart completion time was highly variable between providers working in the same clinic. These patterns were observed across all specialties included in our analysis. Our results suggest a higher level of variability with respect to chart completion when utilizing scribes than previously anticipated.