Electronic phenotypes to distinguish clinician attention to high body mass index, hypertension, lipid disorders, fatty liver and diabetes in pediatric primary care: Diagnostic accuracy of electronic phenotypes compared to masked comprehensive chart review.

BACKGROUND/OBJECTIVES: Electronic phenotyping is a method of using electronic-health-record (EHR) data to automate identifying a patient/population with a characteristic of interest. This study determines validity of using EHR data of children with overweight/obesity to electronically phenotype evidence of clinician 'attention' to high body mass index (BMI) and each of four distinct comorbidities. METHODS: We built five electronic phenotypes classifying 2-18-year-old children with overweight/obesity (n = 17,397) by electronic/health-record evidence of distinct attention to high body mass index, hypertension, lipid disorders, fatty liver, and prediabetes/diabetes. We reviewed, selected and cross-checked random charts to define items clinicians select in EHRs to build problem lists, and to order medications, laboratory tests and referrals to electronically classify attention to overweight/obesity and each comorbidity. Operating characteristics of each clinician-attention phenotype were determined by comparing comprehensive chart review by reviewers masked to electronic classification who adjudicated evidence of clinician attention to high BMI and each comorbidity. RESULTS: In a random sample of 817 visit-records reviewed/coded, specificity of each electronic phenotype is 99%-100% (with PPVs ranging from 96.8% for prediabetes/diabetes to 100% for dyslipidemia and hypertension). Sensitivities of the attention classifications range from 69% for hypertension (NPV, 98.9%) to 84.7% for high-BMI attention (NPV, 92.3%). CONCLUSIONS: Electronic phenotypes for clinician attention to overweight/obesity and distinct comorbidities are highly specific, with moderate (BMI) to modest (each comorbidity) sensitivity. The high specificity supports using phenotypes to identify children with prior high-BMI/comorbidity attention.

Public perception of COVID-19 vaccines through analysis of Twitter content and users.

BACKGROUND: With the global continuation of the COVID-19 pandemic, the large-scale administration of a SARS-CoV-2 vaccine is crucial to achieve herd immunity and curtail further spread of the virus, but success is contingent on public understanding and vaccine uptake. We aim to understand public perception about vaccines for COVID-19 through the wide-scale, organic discussion on Twitter. METHODS: This cross-sectional observational study included Twitter posts matching the search criteria (('covid*' OR 'coronavirus') AND 'vaccine') posted during vaccine development from February 1st through December 11th, 2020. These COVID-19 vaccine related posts were analyzed with topic modeling, sentiment and emotion analysis, and demographic inference of users to provide insight into the evolution of public attitudes throughout the study period. FINDINGS: We evaluated 2,287,344 English tweets from 948,666 user accounts. Individuals represented 87.9 % (n = 834,224) of user accounts. Of individuals, men (n = 560,824) outnumbered women (n = 273,400) by 2:1 and 39.5 % (n = 329,776) of individuals were ≥40 years old. Daily mean sentiment fluctuated congruent with news events, but overall trended positively. Trust, anticipation, and fear were the three most predominant emotions; while fear was the most predominant emotion early in the study period, trust outpaced fear from April 2020 onward. Fear was more prevalent in tweets by individuals (26.3 % vs. organizations 19.4 %; p < 0.001), specifically among women (28.4 % vs. males 25.4 %; p < 0.001). Multiple topics had a monthly trend towards more positive sentiment. Tweets comparing COVID-19 to the influenza vaccine had strongly negative early sentiment but improved over time. INTERPRETATION: This study successfully explores sentiment, emotion, topics, and user demographics to elucidate important trends in public perception about COVID-19 vaccines. While public perception trended positively over the study period, some trends, especially within certain topic and demographic clusters, are concerning for COVID-19 vaccine hesitancy. These insights can provide targets for educational interventions and opportunity for continued real-time monitoring.

District-Level Universal Masking Policies and COVID-19 Incidence During the First 8 Weeks of School in Texas.

Texas discontinued state-sponsored business restrictions and mask mandates on March 10, 2021, and mandated that no government officials, including public school officials, may implement mask requirements even in areas where COVID-19 hospitalizations comprised more than 15% of hospitalizations. Nonetheless, some public school districts began the 2021-2022 school year with mask mandates in place. We used quasi-experimental methods to analyze the impact of school mask mandates, which appear to have resulted in approximately 40 fewer student cases per week in the first eight weeks of school. (Am J Public Health. 2022;112(6):871-875. https://doi.org/10.2105/AJPH.2022.306769).

Rolling Up the Sleeve: Equitable, Efficient, and Safe COVID-19 Mass Immunization for Academic Medical Center Employees.

BACKGROUND: Novel coronavirus disease 2019 (COVID-19) vaccine administration has faced distribution barriers across the United States. We sought to delineate our vaccine delivery experience in the first week of vaccine availability, and our effort to prioritize employees based on risk with a goal of providing an efficient infrastructure to optimize speed and efficiency of vaccine delivery while minimizing risk of infection during the immunization process. OBJECTIVE: This article aims to evaluate an employee prioritization/invitation/scheduling system, leveraging an integrated electronic health record patient portal framework for employee COVID-19 immunizations at an academic medical center. METHODS: We conducted an observational cross-sectional study during January 2021 at a single urban academic center. All employees who met COVID-19 allocation vaccine criteria for phase 1a.1 to 1a.4 were included. We implemented a prioritization/invitation/scheduling framework and evaluated time from invitation to scheduling as a proxy for vaccine interest and arrival to vaccine administration to measure operational throughput. RESULTS: We allotted vaccines for 13,753 employees but only 10,662 employees with an active patient portal account received an invitation. Of those with an active account, 6,483 (61%) scheduled an appointment and 6,251 (59%) were immunized in the first 7 days. About 66% of invited providers were vaccinated in the first 7 days. In contrast, only 41% of invited facility/food service employees received the first dose of the vaccine in the first 7 days (p < 0.001). At the vaccination site, employees waited 5.6 minutes (interquartile range [IQR]: 3.9-8.3) from arrival to vaccination. CONCLUSION: We developed a system of early COVID-19 vaccine prioritization and administration in our health care system. We saw strong early acceptance in those with proximal exposure to COVID-19 but noticed significant difference in the willingness of different employee groups to receive the vaccine.

COVID-19 Mass Vaccination Resource Calculator.

BACKGROUND: Despite the recent emergency use authorization of two vaccines for the prevention of the 2019 novel coronavirus (COVID-19) disease, vaccination rates are lower than expected. Vaccination efforts may be hampered by supply, delivery, storage, patient prioritization, administration infrastructure or logistics problems. To address the last issue, our institution is sharing publically a calculator to optimize the management of staffing and facility resources in an outpatient mass vaccination effort. OBJECTIVE: By sharing our calculator locally and through this paper, we aim to help health organizations administering vaccines optimize resource allocation while maximizing efficiency. METHODS: Our calculator determines the maximum number of vaccinations that can be administered per hour, the number of check-in staff (clerks) needed, the number of vaccination staff (nurses) needed, and the required room capacity needed for the vaccination and the mandatory 15-minute observation period after inoculation. RESULTS: We provide a functional version of the calculator, allowing users to replicate the calculation for their own vaccine events. CONCLUSION: An efficient and organized vaccination program is critical to halting the spread of COVID-19. By sharing this calculator, it is our hope that other organizations may use it to facilitate rapid and efficient vaccination.

Managing Pandemics with Health Informatics: Successes and Challenges.

INTRODUCTION: The novel COVID-19 pandemic struck the world unprepared. This keynote outlines challenges and successes using data to inform providers, government officials, hospitals, and patients in a pandemic. METHODS: The authors outline the data required to manage a novel pandemic including their potential uses by governments, public health organizations, and individuals. RESULTS: An extensive discussion on data quality and on obstacles to collecting data is followed by examples of successes in clinical care, contact tracing, and forecasting. Generic local forecast model development is reviewed followed by ethical consideration around pandemic data. We leave the reader with thoughts on the next inevitable outbreak and lessons learned from the COVID-19 pandemic. CONCLUSION: COVID-19 must be a lesson for the future to direct us to better planning and preparing to manage the next pandemic with health informatics.

Understanding public perception of coronavirus disease 2019 (COVID-19) social distancing on Twitter.

OBJECTIVE: Social distancing policies are key in curtailing severe acute respiratory coronavirus virus 2 (SARS-CoV-2) spread, but their effectiveness is heavily contingent on public understanding and collective adherence. We studied public perception of social distancing through organic, large-scale discussion on Twitter. DESIGN: Retrospective cross-sectional study. METHODS: Between March 27 and April 10, 2020, we retrieved English-only tweets matching two trending social distancing hashtags, #socialdistancing and #stayathome. We analyzed the tweets using natural language processing and machine-learning models, and we conducted a sentiment analysis to identify emotions and polarity. We evaluated the subjectivity of tweets and estimated the frequency of discussion of social distancing rules. We then identified clusters of discussion using topic modeling and associated sentiments. RESULTS: We studied a sample of 574,903 tweets. For both hashtags, polarity was positive (mean, 0.148; SD, 0.290); only 15% of tweets had negative polarity. Tweets were more likely to be objective (median, 0.40; IQR, 0-0.6) with ~30% of tweets labeled as completely objective (labeled as 0 in range from 0 to 1). Approximately half of tweets (50.4%) primarily expressed joy and one-fifth expressed fear and surprise. Each correlated well with topic clusters identified by frequency including leisure and community support (ie, joy), concerns about food insecurity and quarantine effects (ie, fear), and unpredictability of coronavirus disease 2019 (COVID-19) and its implications (ie, surprise). CONCLUSIONS: Considering the positive sentiment, preponderance of objective tweets, and topics supporting coping mechanisms, we concluded that Twitter users generally supported social distancing in the early stages of their implementation.

Derivation With Internal Validation of a Multivariable Predictive Model to Predict COVID-19 Test Results in Emergency Department Patients.

OBJECTIVES: The COVID-19 pandemic has placed acute care providers in demanding situations in predicting disease given the clinical variability, desire to cohort patients, and high variance in testing availability. An approach to stratifying patients by likelihood of disease based on rapidly available emergency department (ED) clinical data would offer significant operational and clinical value. The purpose of this study was to develop and internally validate a predictive model to aid in the discrimination of patients undergoing investigation for COVID-19. METHODS: All patients greater than 18 years presenting to a single academic ED who were tested for COVID-19 during this index ED evaluation were included. Outcome was defined as the result of COVID-19 polymerase chain reaction (PCR) testing during the index visit or any positive result within the following 7 days. Variables included chest radiograph interpretation, disease-specific screening questions, and laboratory data. Three models were developed with a split-sample approach to predict outcome of the PCR test utilizing logistic regression, random forest, and gradient-boosted decision tree methods. Model discrimination was evaluated comparing area under the receiver operator curve (AUC) and point statistics at a predefined threshold. RESULTS: A total of 1,026 patients were included in the study collected between March and April 2020. Overall, there was disease prevalence of 9.6% in the population under study during this time frame. The logistic regression model was found to have an AUC of 0.89 (95% confidence interval [CI] = 0.84 to 0.94) when including four features: exposure history, temperature, white blood cell count (WBC), and chest radiograph result. Random forest method resulted in AUC of 0.86 (95% CI = 0.79 to 0.92) and gradient boosting had an AUC of 0.85 (95% CI = 0.79 to 0.91). With a consistently held negative predictive value, the logistic regression model had a positive predictive value of 0.29 (0.2-0.39) compared to 0.2 (0.14-0.28) for random forest and 0.22 (0.15-0.3) for the gradient-boosted method. CONCLUSION: The derived predictive models offer good discriminating capacity for COVID-19 disease and provide interpretable and usable methods for those providers caring for these patients at the important crossroads of the community and the health system. We found utilization of the logistic regression model utilizing exposure history, temperature, WBC, and chest X-ray result had the greatest discriminatory capacity with the most interpretable model. Integrating a predictive model-based approach to COVID-19 testing decisions and patient care pathways and locations could add efficiency and accuracy to decrease uncertainty.

User stories as lightweight requirements for agile clinical decision support development.

OBJECTIVE: We sought to demonstrate applicability of user stories, progressively elaborated by testable acceptance criteria, as lightweight requirements for agile development of clinical decision support (CDS). MATERIALS AND METHODS: User stories employed the template: As a [type of user], I want [some goal] so that [some reason]. From the "so that" section, CDS benefit measures were derived. Detailed acceptance criteria were elaborated through ensuing conversations. We estimated user story size with "story points," and depicted multiple user stories with a use case diagram or feature breakdown structure. Large user stories were split to fit into 2-week iterations. RESULTS: One example user story was: As a rheumatologist, I want to be advised if my patient with rheumatoid arthritis is not on a disease-modifying anti-rheumatic drug (DMARD), so that they receive optimal therapy and can experience symptom improvement. This yielded a process measure (DMARD use), and an outcome measure (Clinical Disease Activity Index). Following implementation, the DMARD nonuse rate decreased from 3.7% to 1.4%. Patients with a high Clinical Disease Activity Index improved from 13.7% to 7%. For a thromboembolism prevention CDS project, diagrams organized multiple user stories. DISCUSSION: User stories written in the clinician's voice aid CDS governance and lead naturally to measures of CDS effectiveness. Estimation of relative story size helps plan CDS delivery dates. User stories prove to be practical even on larger projects. CONCLUSIONS: User stories concisely communicate the who, what, and why of a CDS request, and serve as lightweight requirements for agile development to meet the demand for increasingly diverse CDS.

Correction: SNOMED CT Concept Hierarchies for Computable Clinical Phenotypes From Electronic Health Record Data: Comparison of Intensional Versus Extensional Value Sets.

[This corrects the article DOI: 10.2196/11487.].

SNOMED CT Concept Hierarchies for Computable Clinical Phenotypes From Electronic Health Record Data: Comparison of Intensional Versus Extensional Value Sets.

BACKGROUND: Defining clinical phenotypes from electronic health record (EHR)-derived data proves crucial for clinical decision support, population health endeavors, and translational research. EHR diagnoses now commonly draw from a finely grained clinical terminology-either native SNOMED CT or a vendor-supplied terminology mapped to SNOMED CT concepts as the standard for EHR interoperability. Accordingly, electronic clinical quality measures (eCQMs) increasingly define clinical phenotypes with SNOMED CT value sets. The work of creating and maintaining list-based value sets proves daunting, as does insuring that their contents accurately represent the clinically intended condition. OBJECTIVE: The goal of the research was to compare an intensional (concept hierarchy-based) versus extensional (list-based) value set approach to defining clinical phenotypes using SNOMED CT-encoded data from EHRs by evaluating value set conciseness, time to create, and completeness. METHODS: Starting from published Centers for Medicare and Medicaid Services (CMS) high-priority eCQMs, we selected 10 clinical conditions referenced by those eCQMs. For each, the published SNOMED CT list-based (extensional) value set was downloaded from the Value Set Authority Center (VSAC). Ten corresponding SNOMED CT hierarchy-based intensional value sets for the same conditions were identified within our EHR. From each hierarchy-based intensional value set, an exactly equivalent full extensional value set was derived enumerating all included descendant SNOMED CT concepts. Comparisons were then made between (1) VSAC-downloaded list-based (extensional) value sets, (2) corresponding hierarchy-based intensional value sets for the same conditions, and (3) derived list-based (extensional) value sets exactly equivalent to the hierarchy-based intensional value sets. Value set conciseness was assessed by the number of SNOMED CT concepts needed for definition. Time to construct the value sets for local use was measured. Value set completeness was assessed by comparing contents of the downloaded extensional versus intensional value sets. Two measures of content completeness were made: for individual SNOMED CT concepts and for the mapped diagnosis clinical terms available for selection within the EHR by clinicians. RESULTS: The 10 hierarchy-based intensional value sets proved far simpler and faster to construct than exactly equivalent derived extensional value set lists, requiring a median 3 versus 78 concepts to define and 5 versus 37 minutes to build. The hierarchy-based intensional value sets also proved more complete: in comparison, the 10 downloaded 2018 extensional value sets contained a median of just 35% of the intensional value sets' SNOMED CT concepts and 65% of mapped EHR clinical terms. CONCLUSIONS: In the EHR era, defining conditions preferentially should employ SNOMED CT concept hierarchy-based (intensional) value sets rather than extensional lists. By doing so, clinical guideline and eCQM authors can more readily engage specialists in vetting condition subtypes to include and exclude, and streamline broad EHR implementation of condition-specific decision support promoting guideline adherence for patient benefit.

SNOMED CT Concept Hierarchies for Sharing Definitions of Clinical Conditions Using Electronic Health Record Data.

BACKGROUND: Defining clinical conditions from electronic health record (EHR) data underpins population health activities, clinical decision support, and analytics. In an EHR, defining a condition commonly employs a diagnosis value set or "grouper." For constructing value sets, Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) offers high clinical fidelity, a hierarchical ontology, and wide implementation in EHRs as the standard interoperability vocabulary for problems. OBJECTIVE: This article demonstrates a practical approach to defining conditions with combinations of SNOMED CT concept hierarchies, and evaluates sharing of definitions for clinical and analytic uses. METHODS: We constructed diagnosis value sets for EHR patient registries using SNOMED CT concept hierarchies combined with Boolean logic, and shared them for clinical decision support, reporting, and analytic purposes. RESULTS: A total of 125 condition-defining "standard" SNOMED CT diagnosis value sets were created within our EHR. The median number of SNOMED CT concept hierarchies needed was only 2 (25th-75th percentiles: 1-5). Each value set, when compiled as an EHR diagnosis grouper, was associated with a median of 22 International Classification of Diseases (ICD)-9 and ICD-10 codes (25th-75th percentiles: 8-85) and yielded a median of 155 clinical terms available for selection by clinicians in the EHR (25th-75th percentiles: 63-976). Sharing of standard groupers for population health, clinical decision support, and analytic uses was high, including 57 patient registries (with 362 uses of standard groupers), 132 clinical decision support records, 190 rules, 124 EHR reports, 125 diagnosis dimension slicers for self-service analytics, and 111 clinical quality measure calculations. Identical SNOMED CT definitions were created in an EHR-agnostic tool enabling application across disparate organizations and EHRs. CONCLUSION: SNOMED CT-based diagnosis value sets are simple to develop, concise, understandable to clinicians, useful in the EHR and for analytics, and shareable. Developing curated SNOMED CT hierarchy-based condition definitions for public use could accelerate cross-organizational population health efforts, "smarter" EHR feature configuration, and clinical-translational research employing EHR-derived data.

Agile Acceptance Test-Driven Development of Clinical Decision Support Advisories: Feasibility of Using Open Source Software.

BACKGROUND: Moving to electronic health records (EHRs) confers substantial benefits but risks unintended consequences. Modern EHRs consist of complex software code with extensive local configurability options, which can introduce defects. Defects in clinical decision support (CDS) tools are surprisingly common. Feasible approaches to prevent and detect defects in EHR configuration, including CDS tools, are needed. In complex software systems, use of test-driven development and automated regression testing promotes reliability. Test-driven development encourages modular, testable design and expanding regression test coverage. Automated regression test suites improve software quality, providing a "safety net" for future software modifications. Each automated acceptance test serves multiple purposes, as requirements (prior to build), acceptance testing (on completion of build), regression testing (once live), and "living" design documentation. Rapid-cycle development or "agile" methods are being successfully applied to CDS development. The agile practice of automated test-driven development is not widely adopted, perhaps because most EHR software code is vendor-developed. However, key CDS advisory configuration design decisions and rules stored in the EHR may prove amenable to automated testing as "executable requirements." OBJECTIVE: We aimed to establish feasibility of acceptance test-driven development of clinical decision support advisories in a commonly used EHR, using an open source automated acceptance testing framework (FitNesse). METHODS: Acceptance tests were initially constructed as spreadsheet tables to facilitate clinical review. Each table specified one aspect of the CDS advisory's expected behavior. Table contents were then imported into a test suite in FitNesse, which queried the EHR database to automate testing. Tests and corresponding CDS configuration were migrated together from the development environment to production, with tests becoming part of the production regression test suite. RESULTS: We used test-driven development to construct a new CDS tool advising Emergency Department nurses to perform a swallowing assessment prior to administering oral medication to a patient with suspected stroke. Test tables specified desired behavior for (1) applicable clinical settings, (2) triggering action, (3) rule logic, (4) user interface, and (5) system actions in response to user input. Automated test suite results for the "executable requirements" are shown prior to building the CDS alert, during build, and after successful build. CONCLUSIONS: Automated acceptance test-driven development and continuous regression testing of CDS configuration in a commercial EHR proves feasible with open source software. Automated test-driven development offers one potential contribution to achieving high-reliability EHR configuration. Vetting acceptance tests with clinicians elicits their input on crucial configuration details early during initial CDS design and iteratively during rapid-cycle optimization.

Agile Co-Development for Clinical Adoption and Adaptation of Innovative Technologies.

Even the most innovative healthcare technologies provide patient benefits only when adopted by clinicians and/or patients in actual practice. Yet realizing optimal positive impact from a new technology for the widest range of individuals who would benefit remains elusive. In software and new product development, iterative rapid-cycle "agile" methods more rapidly provide value, mitigate failure risks, and adapt to customer feedback. Co-development between builders and customers is a key agile principle. But how does one accomplish co-development with busy clinicians? In this paper, we discuss four practical agile co-development practices found helpful clinically: (1) User stories for lightweight requirements; (2) Time-boxed development for collaborative design and prompt course correction; (3) Automated acceptance test driven development, with clinician-vetted specifications; and (4) Monitoring of clinician interactions after release, for rapid-cycle product adaptation and evolution. In the coming wave of innovation in healthcare apps ushered in by open APIs to EHRs, learning rapidly what new product features work well for clinicians and patients will become even more crucial.