Implementation of pharmacogenomic clinical decision support for health systems: a cost-utility analysis.

We constructed a cost-effectiveness model to assess the clinical and economic value of a CDS alert program that provides pharmacogenomic (PGx) testing results, compared to no alert program in acute coronary syndrome (ACS) and atrial fibrillation (AF), from a health system perspective. We defaulted that 20% of 500,000 health-system members between the ages of 55 and 65 received PGx testing for CYP2C19 (ACS-clopidogrel) and CYP2C9, CYP4F2 and VKORC1 (AF-warfarin) annually. Clinical events, costs, and quality-adjusted life years (QALYs) were calculated over 20 years with an annual discount rate of 3%. In total, 3169 alerts would be fired. The CDS alert program would help avoid 16 major clinical events and 6 deaths for ACS; and 2 clinical events and 0.9 deaths for AF. The incremental cost-effectiveness ratio was $39,477/QALY. A PGx-CDS alert program was cost-effective, under a willingness-to-pay threshold of $100,000/QALY gained, compared to no alert program.

The Feelings About genomiC Testing Results (FACToR) Questionnaire: Development and Preliminary Validation.

The purpose of this study was to develop a brief instrument, the Feelings About genomiC Testing Results (FACToR), to measure the psychosocial impact of returning genomic findings to patients in research and clinical practice. To create the FACToR, we modified and augmented the Multidimensional Impact of Cancer Risk Assessment (MICRA) questionnaire based on findings from a literature review, two focus groups (N = 12), and cognitive interviews (N = 6). We evaluated data from 122 participants referred for evaluation for inherited colorectal cancer or polyposis from the New EXome Technology in (NEXT) Medicine Study, an RCT of exome sequencing versus usual care. We assessed floor and ceiling effects of each item, conducted principal component analysis to identify subscales, and evaluated each subscale's internal consistency, test-retest reliability, and construct validity. After excluding items that were ambiguous or demonstrated floor or ceiling effects, 12 items forming four distinct subscales were retained for further analysis: negative emotions, positive feelings, uncertainty, and privacy concerns. All four showed good internal consistency (0.66-0.78) and test-retest reliability (0.65-0.91). The positive feelings and the uncertainty subscales demonstrated known-group validity. The 12-item FACToR with four subscales shows promising psychometric properties on preliminary evaluation in a limited sample and needs to be evaluated in other populations.

Development of clinical decision support alerts for pharmacogenomic incidental findings from exome sequencing.

PURPOSE: Electronic health records (EHRs) and their associated decision support tools are potentially important means of disseminating a patient's pharmacogenomic profile to his or her health-care providers. We sought to create a proof-of-concept decision support alert system generated from pharmacogenomic incidental findings from exome sequencing. METHODS: A pipeline for alerts from exome sequencing tests was created for patients in the New EXome Technology in (NEXT) Medicine study at the University of Washington. Decision support rules using discrete, machine-readable incidental finding results were programmed into a commercial EHR rules engine. An evaluation plan to monitor the alerts in real medical interactions was established. RESULTS: Alerts were created for 48 actionable pharmacogenomic variants in 11 genes and were launched on 24 September 2014 for University of Washington inpatient care. Of the 94 participants enrolled in the NEXT Medicine study, 49 had one or more pharmacogenomic variants identified for return. CONCLUSION: Reflections on the process reveal that while incidental findings can be used to generate decision support alerts, substantial resources are required to ensure that each alert is consistent with rapidly evolving pharmacogenomic literature and is customized to fit in the clinical workflow unique to each incidental finding.

Making pharmacogenomic-based prescribing alerts more effective: A scenario-based pilot study with physicians.

To facilitate personalized drug dosing (PDD), this pilot study explored the communication effectiveness and clinical impact of using a prototype clinical decision support (CDS) system embedded in an electronic health record (EHR) to deliver pharmacogenomic (PGx) information to physicians. We employed a conceptual framework and measurement model to access the impact of physician characteristics (previous experience, awareness, relative advantage, perceived usefulness), technology characteristics (methods of implementation-semi-active/active, actionability-low/high) and a task characteristic (drug prescribed) on communication effectiveness (usefulness, confidence in prescribing decision), and clinical impact (uptake, prescribing intent, change in drug dosing). Physicians performed prescribing tasks using five simulated clinical case scenarios, presented in random order within the prototype PGx-CDS system. Twenty-two physicians completed the study. The proportion of physicians that saw a relative advantage to using PGx-CDS was 83% at the start and 94% at the conclusion of our study. Physicians used semi-active alerts 74-88% of the time. There was no association between previous experience with, awareness of, and belief in a relative advantage of using PGx-CDS and improved uptake. The proportion of physicians reporting confidence in their prescribing decisions decreased significantly after using the prototype PGx-CDS system (p=0.02). Despite decreases in confidence, physicians perceived a relative advantage to using PGx-CDS, viewed semi-active alerts on most occasions, and more frequently changed doses toward doses supported by published evidence. Specifically, sixty-five percent of physicians reduced their dosing, significantly for capecitabine (p=0.002) and mercaptopurine/thioguanine (p=0.03). These findings suggest a need to improve our prototype such that PGx CDS content is more useful and delivered in a way that improves physician's confidence in their prescribing decisions. The greatest increases in communication effectiveness and clinical impact of PGx-CDS are likely to be realized through continued focus on content, content delivery, and tailoring to physician characteristics.

Refining the structure and content of clinical genomic reports.

To effectively articulate the results of exome and genome sequencing we refined the structure and content of molecular test reports. To communicate results of a randomized control trial aimed at the evaluation of exome sequencing for clinical medicine, we developed a structured narrative report. With feedback from genetics and non-genetics professionals, we developed separate indication-specific and incidental findings reports. Standard test report elements were supplemented with research study-specific language, which highlighted the limitations of exome sequencing and provided detailed, structured results, and interpretations. The report format we developed to communicate research results can easily be transformed for clinical use by removal of research-specific statements and disclaimers. The development of clinical reports for exome sequencing has shown that accurate and open communication between the clinician and laboratory is ideally an ongoing process to address the increasing complexity of molecular genetic testing.

A survey of informatics approaches to whole-exome and whole-genome clinical reporting in the electronic health record.

PURPOSE: Genome-scale clinical sequencing is being adopted more broadly in medical practice. The National Institutes of Health developed the Clinical Sequencing Exploratory Research (CSER) program to guide implementation and dissemination of best practices for the integration of sequencing into clinical care. This study describes and compares the state of the art of incorporating whole-exome and whole-genome sequencing results into the electronic health record, including approaches to decision support across the six current CSER sites. METHODS: The CSER Medical Record Working Group collaboratively developed and completed an in-depth survey to assess the communication of genome-scale data into the electronic health record. We summarized commonalities and divergent approaches. RESULTS: Despite common sequencing platform (Illumina) adoptions, there is a great diversity of approaches to annotation tools and workflow, as well as to report generation. At all sites, reports are human-readable structured documents available as passive decision support in the electronic health record. Active decision support is in early implementation at two sites. CONCLUSION: The parallel efforts across CSER sites in the creation of systems for report generation and integration of reports into the electronic health record, as well as the lack of standardized approaches to interfacing with variant databases to create active clinical decision support, create opportunities for cross-site and vendor collaborations.

Uses of Electronic Health Record Data to Measure the Clinical Learning Environment of Graduate Medical Education Trainees: A Systematic Review.

PURPOSE: This study systematically reviews the uses of electronic health record (EHR) data to measure graduate medical education (GME) trainee competencies. METHOD: In January 2022, the authors conducted a systematic review of original research in MEDLINE from database start to December 31, 2021. The authors searched for articles that used the EHR as their data source and in which the individual GME trainee was the unit of observation and/or unit of analysis. The database query was intentionally broad because an initial survey of pertinent articles identified no unifying Medical Subject Heading terms. Articles were coded and clustered by theme and Accreditation Council for Graduate Medical Education (ACGME) core competency. RESULTS: The database search yielded 3,540 articles, of which 86 met the study inclusion criteria. Articles clustered into 16 themes, the largest of which were trainee condition experience (17 articles), work patterns (16 articles), and continuity of care (12 articles). Five of the ACGME core competencies were represented (patient care and procedural skills, practice-based learning and improvement, systems-based practice, medical knowledge, and professionalism). In addition, 25 articles assessed the clinical learning environment. CONCLUSIONS: This review identified 86 articles that used EHR data to measure individual GME trainee competencies, spanning 16 themes and 6 competencies and revealing marked between-trainee variation. The authors propose a digital learning cycle framework that arranges sequentially the uses of EHR data within the cycle of clinical experiential learning central to GME. Three technical components necessary to unlock the potential of EHR data to improve GME are described: measures, attribution, and visualization. Partnerships between GME programs and informatics departments will be pivotal in realizing this opportunity.

Auditing Learned Associations in Deep Learning Approaches to Extract Race and Ethnicity from Clinical Text.

Complete and accurate race and ethnicity (RE) patient information is important for many areas of biomedical informatics research, such as defining and characterizing cohorts, performing quality assessments, and identifying health inequities. Patient-level RE data is often inaccurate or missing in structured sources, but can be supplemented through clinical notes and natural language processing (NLP). While NLP has made many improvements in recent years with large language models, bias remains an often-unaddressed concern, with research showing that harmful and negative language is more often used for certain racial/ethnic groups than others. We present an approach to audit the learned associations of models trained to identify RE information in clinical text by measuring the concordance between model-derived salient features and manually identified RE-related spans of text. We show that while models perform well on the surface, there exist concerning learned associations and potential for future harms from RE-identification models if left unaddressed.

Learning virulent proteins from integrated query networks.

BACKGROUND: Methods of weakening and attenuating pathogens' abilities to infect and propagate in a host, thus allowing the natural immune system to more easily decimate invaders, have gained attention as alternatives to broad-spectrum targeting approaches. The following work describes a technique to identifying proteins involved in virulence by relying on latent information computationally gathered across biological repositories, applicable to both generic and specific virulence categories. RESULTS: A lightweight method for data integration is used, which links information regarding a protein via a path-based query graph. A method of weighting is then applied to query graphs that can serve as input to various statistical classification methods for discrimination, and the combined usage of both data integration and learning methods are tested against the problem of both generalized and specific virulence function prediction. CONCLUSIONS: This approach improves coverage of functional data over a protein. Moreover, while depending largely on noisy and potentially non-curated data from public sources, we find it outperforms other techniques to identification of general virulence factors and baseline remote homology detection methods for specific virulence categories.

A survey of informatics platforms that enable distributed comparative effectiveness research using multi-institutional heterogenous clinical data.

Comparative effectiveness research (CER) has the potential to transform the current health care delivery system by identifying the most effective medical and surgical treatments, diagnostic tests, disease prevention methods, and ways to deliver care for specific clinical conditions. To be successful, such research requires the identification, capture, aggregation, integration, and analysis of disparate data sources held by different institutions with diverse representations of the relevant clinical events. In an effort to address these diverse demands, there have been multiple new designs and implementations of informatics platforms that provide access to electronic clinical data and the governance infrastructure required for interinstitutional CER. The goal of this manuscript is to help investigators understand why these informatics platforms are required and to compare and contrast 6 large-scale, recently funded, CER-focused informatics platform development efforts. We utilized an 8-dimension, sociotechnical model of health information technology to help guide our work. We identified 6 generic steps that are necessary in any distributed, multi-institutional CER project: data identification, extraction, modeling, aggregation, analysis, and dissemination. We expect that over the next several years these projects will provide answers to many important, and heretofore unanswerable, clinical research questions.

Characterizing Data Discovery and End-User Computing Needs in Clinical Translational Science.

In this paper, the authors present the results of a qualitative case-study seeking to characterize data discovery needs and barriers of principal investigators and research support staff in clinical translational science. Several implications for designing and implementing translational research systems have emerged through the authors' analysis. The results also illustrate the benefits of forming early partnerships with scientists to better understand their workflow processes and end-user computing practices in accessing data for research. The authors use this user-centered, iterative development approach to guide the implementation and extension of i2b2, a system they have adapted to support cross-institutional aggregate anonymized clinical data querying. With ongoing evaluation, the goal is to maximize the utility and extension of this system and develop an interface that appropriately fits the swiftly evolving needs of clinical translational scientists.

A Novel Food Record App for Dietary Assessments Among Older Adults With Type 2 Diabetes: Development and Usability Study.

BACKGROUND: More than 1 in 4 people in the United States aged 65 years and older have type 2 diabetes. For diabetes care, medical nutrition therapy is recommended as a clinically effective intervention. Previous researchers have developed and validated dietary assessment methods using images of food items to improve the accuracy of self-reporting over traditional methods. Nevertheless, little is known about the usability of image-assisted dietary assessment methods for older adults with diabetes. OBJECTIVE: The aims of this study were (1) to create a food record app for dietary assessments (FRADA) that would support image-assisted dietary assessments, and (2) to evaluate the usability of FRADA for older adults with diabetes. METHODS: For the development of FRADA, we identified design principles that address the needs of older adults and implemented three fundamental tasks required for image-assisted dietary assessments: capturing, viewing, and transmitting images of food based on the design principles. For the usability assessment of FRADA, older adults aged 65 to 80 years (11 females and 3 males) were assigned to interact with FRADA in a lab-based setting. Participants' opinions of FRADA and its usability were determined by a follow-up survey and interview. As an evaluation indicator of usability, the responses to the survey, including an after-scenario questionnaire, were analyzed. Qualitative data from the interviews confirmed the responses to the survey. RESULTS: We developed a smartphone app that enables older adults with diabetes to capture, view, and transmit images of food items they consumed. The findings of this study showed that FRADA and its instructions for capturing, viewing, and transmitting images of food items were usable for older adults with diabetes. The survey showed that participants found FRADA easy to use and would consider using FRADA daily. The analysis of the qualitative data from interviews revealed multiple categories, such as the usability of FRADA, potential benefits of using FRADA, potential features to be added to FRADA, and concerns of older adults with diabetes regarding interactions with FRADA. CONCLUSIONS: This study demonstrates in a lab-based setting not only the usability of FRADA by older adults with diabetes but also potential opportunities using FRADA in real-world settings. The findings suggest implications for creating a smartphone app for an image-assisted dietary assessment. Future work still remains to evaluate the feasibility and validity of FRADA with multiple stakeholders, including older adults with diabetes and dietitians.

Using a Constraint-Based Method to Identify Chronic Disease Patients Who Are Apt to Obtain Care Mostly Within a Given Health Care System: Retrospective Cohort Study.

BACKGROUND: For several major chronic diseases including asthma, chronic obstructive pulmonary disease, chronic kidney disease, and diabetes, a state-of-the-art method to avert poor outcomes is to use predictive models to identify future high-cost patients for preemptive care management interventions. Frequently, an American patient obtains care from multiple health care systems, each managed by a distinct institution. As the patient's medical data are spread across these health care systems, none has complete medical data for the patient. The task of building models to predict an individual patient's cost is currently thought to be impractical with incomplete data, which limits the use of care management to improve outcomes. Recently, we developed a constraint-based method to identify patients who are apt to obtain care mostly within a given health care system. Our method was shown to work well for the cohort of all adult patients at the University of Washington Medicine for a 6-month follow-up period. It is unknown how well our method works for patients with various chronic diseases and over follow-up periods of different lengths, and subsequently, whether it is reasonable to perform this predictive modeling task on the subset of patients pinpointed by our method. OBJECTIVE: To understand our method's potential to enable this predictive modeling task on incomplete medical data, this study assesses our method's performance at the University of Washington Medicine on 5 subgroups of adult patients with major chronic diseases and over follow-up periods of 2 different lengths. METHODS: We used University of Washington Medicine data for all adult patients who obtained care at the University of Washington Medicine in 2018 and PreManage data containing usage information from all hospitals in Washington state in 2019. We evaluated our method's performance over the follow-up periods of 6 months and 12 months on 5 patient subgroups separately-asthma, chronic kidney disease, type 1 diabetes, type 2 diabetes, and chronic obstructive pulmonary disease. RESULTS: Our method identified 21.81% (3194/14,644) of University of Washington Medicine adult patients with asthma. Around 66.75% (797/1194) and 67.13% (1997/2975) of their emergency department visits and inpatient stays took place within the University of Washington Medicine system in the subsequent 6 months and in the subsequent 12 months, respectively, approximately double the corresponding percentage for all University of Washington Medicine adult patients with asthma. The performance for adult patients with chronic kidney disease, adult patients with chronic obstructive pulmonary disease, adult patients with type 1 diabetes, and adult patients with type 2 diabetes was reasonably similar to that for adult patients with asthma. CONCLUSIONS: For each of the 5 chronic diseases most relevant to care management, our method can pinpoint a reasonably large subset of patients who are apt to obtain care mostly within the University of Washington Medicine system. This opens the door to building models to predict an individual patient's cost on incomplete data, which was formerly deemed impractical. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/13783.

Feasibility of incorporating genomic knowledge into electronic medical records for pharmacogenomic clinical decision support.

In pursuing personalized medicine, pharmacogenomic (PGx) knowledge may help guide prescribing drugs based on a person's genotype. Here we evaluate the feasibility of incorporating PGx knowledge, combined with clinical data, to support clinical decision-making by: 1) analyzing clinically relevant knowledge contained in PGx knowledge resources; 2) evaluating the feasibility of a rule-based framework to support formal representation of clinically relevant knowledge contained in PGx knowledge resources; and, 3) evaluating the ability of an electronic medical record/electronic health record (EMR/EHR) to provide computable forms of clinical data needed for PGx clinical decision support. Findings suggest that the PharmGKB is a good source for PGx knowledge to supplement information contained in FDA approved drug labels. Furthermore, we found that with supporting knowledge (e.g. IF age <18 THEN patient is a child), sufficient clinical data exists in University of Washington's EMR systems to support 50% of PGx knowledge contained in drug labels that could be expressed as rules.

Supporting retrieval of diverse biomedical data using evidence-aware queries.

Though there have been many advances in providing access to linked and integrated biomedical data across repositories, developing methods which allow users to specify ambiguous and exploratory queries over disparate sources remains a challenge to extracting well-curated or diversely-supported biological information. In the following work, we discuss the concepts of data coverage and evidence in the context of integrated sources. We address diverse information retrieval via a simple framework for representing coverage and evidence that operates in parallel with an arbitrary schema, and a language upon which queries on the schema and framework may be executed. We show that this approach is capable of answering questions that require ranged levels of evidence or triangulation, and demonstrate that appropriately-formed queries can significantly improve the level of precision when retrieving well-supported biomedical data.

Evaluation of probabilistic and logical inference for a SNP annotation system.

Genome wide association studies (GWAS) are an important approach to understanding the genetic mechanisms behind human diseases. Single nucleotide polymorphisms (SNPs) are the predominant markers used in genome wide association studies, and the ability to predict which SNPs are likely to be functional is important for both a priori and a posteriori analyses of GWA studies. This article describes the design, implementation and evaluation of a family of systems for the purpose of identifying SNPs that may cause a change in phenotypic outcomes. The methods described in this article characterize the feasibility of combinations of logical and probabilistic inference with federated data integration for both point and regional SNP annotation and analysis. Evaluations of the methods demonstrate the overall strong predictive value of logical, and logical with probabilistic, inference applied to the domain of SNP annotation.

Prescriber and staff perceptions of an electronic prescribing system in primary care: a qualitative assessment.

BACKGROUND: The United States (US) Health Information Technology for Economic and Clinical Health Act of 2009 has spurred adoption of electronic health records. The corresponding meaningful use criteria proposed by the Centers for Medicare and Medicaid Services mandates use of computerized provider order entry (CPOE) systems. Yet, adoption in the US and other Western countries is low and descriptions of successful implementations are primarily from the inpatient setting; less frequently the ambulatory setting. We describe prescriber and staff perceptions of implementation of a CPOE system for medications (electronic- or e-prescribing system) in the ambulatory setting. METHODS: Using a cross-sectional study design, we conducted eight focus groups at three primary care sites in an independent medical group. Each site represented a unique stage of e-prescribing implementation - pre/transition/post. We used a theoretically based, semi-structured questionnaire to elicit physician (n = 17) and staff (n = 53) perceptions of implementation of the e-prescribing system. We conducted a thematic analysis of focus group discussions using formal qualitative analytic techniques (i.e. deductive framework and grounded theory). Two coders independently coded to theoretical saturation and resolved discrepancies through discussions. RESULTS: Ten themes emerged that describe perceptions of e-prescribing implementation: 1) improved availability of clinical information resulted in prescribing efficiencies and more coordinated care; 2) improved documentation resulted in safer care; 3) efficiencies were gained by using fewer paper charts; 4) organizational support facilitated adoption; 5) transition required time; resulted in workload shift to staff; 6) hardware configurations and network stability were important in facilitating workflow; 7) e-prescribing was time-neutral or time-saving; 8) changes in patient interactions enhanced patient care but required education; 9) pharmacy communications were enhanced but required education; 10) positive attitudes facilitated adoption. CONCLUSIONS: Prescribers and staff worked through the transition to successfully adopt e-prescribing, and noted the benefits. Overall impressions were favorable. No one wished to return to paper-based prescribing.

Participatory (re)design of a sociotechnical healthcare delivery system: the Group Health Patient-Centered Medical Home.

This paper describes one organization's interpretation of the Patient-Centered Medical Home concept and the healthcare delivery system that has emerged from their participatory redesign initiative. Group Health, a large integrated healthcare system based in Seattle, Washington, USA initiated a Patient-Centered Medical Home care delivery system transformation in January 2007. Current theories and evidence about the Patient-Centered Medical Home (PCMH), the Chronic Care Model, and effective primary care were interpreted via a facilitated group process and translated into a core set of 5 system design principles. These design principles guided all subsequent system transformation activities. The central organizing principle is supporting and sustaining the patient-primary care physician relationship. The emergent PCMH healthcare delivery system comprises both opportunistic point-of-care and outreach components, many of which leverage and enhance the organization's health information and communication technologies.

Leaf: an open-source, model-agnostic, data-driven web application for cohort discovery and translational biomedical research.

OBJECTIVE: Academic medical centers and health systems are increasingly challenged with supporting appropriate secondary use of clinical data. Enterprise data warehouses have emerged as central resources for these data, but often require an informatician to extract meaningful information, limiting direct access by end users. To overcome this challenge, we have developed Leaf, a lightweight self-service web application for querying clinical data from heterogeneous data models and sources. MATERIALS AND METHODS: Leaf utilizes a flexible biomedical concept system to define hierarchical concepts and ontologies. Each Leaf concept contains both textual representations and SQL query building blocks, exposed by a simple drag-and-drop user interface. Leaf generates abstract syntax trees which are compiled into dynamic SQL queries. RESULTS: Leaf is a successful production-supported tool at the University of Washington, which hosts a central Leaf instance querying an enterprise data warehouse with over 300 active users. Through the support of UW Medicine (https://uwmedicine.org), the Institute of Translational Health Sciences (https://www.iths.org), and the National Center for Data to Health (https://ctsa.ncats.nih.gov/cd2h/), Leaf source code has been released into the public domain at https://github.com/uwrit/leaf. DISCUSSION: Leaf allows the querying of single or multiple clinical databases simultaneously, even those of different data models. This enables fast installation without costly extraction or duplication. CONCLUSIONS: Leaf differs from existing cohort discovery tools because it does not specify a required data model and is designed to seamlessly leverage existing user authentication systems and clinical databases in situ. We believe Leaf to be useful for health system analytics, clinical research data warehouses, precision medicine biobanks, and clinical studies involving large patient cohorts.