The development and validation of a simulation tool for health policy decision making.

Computer simulations have been used to model infectious diseases to examine the outcomes of alternative strategies for managing their spread. Methicillin resistant Staphylococcus aureus (MRSA) skin and soft tissue infections have become prominent in many communities and efforts are underway to reduce the spread of this organism both in hospitals and communities. Currently, there are few tools for policy makers to use to examine the outcome of various choices when making decisions about MRSA. Using the example of MRSA, we describe, in this paper, a rigorous approach for development and validation of a tool that simulates the spread of MRSA infections. We used sensitivity analyses in a novel way and validated the simulation results against local data over time. Our approach for simulation development and validation is generalizeable to simulations of other diseases.

Domains, tasks, and knowledge for health informatics practice: results of a practice analysis.

OBJECTIVE: To develop a comprehensive and current description of what health informatics (HI) professionals do and what they need to know. MATERIALS AND METHODS: Six independent subject-matter expert panels drawn from and representative of HI professionals contributed to the development of a draft HI delineation of practice (DoP). An online survey was distributed to HI professionals to validate the draft DoP. A total of 1011 HI practitioners completed the survey. Survey respondents provided domain, task, knowledge and skill (KS) ratings, qualitative feedback on the completeness of the DoP, and detailed professional background and demographic information. RESULTS: This practice analysis resulted in a validated, comprehensive, and contemporary DoP comprising 5 domains, 74 tasks, and 144 KS statements. DISCUSSION: The HI practice analysis defined "health informatics professionals" to include practitioners with clinical (eg, dentistry, nursing, pharmacy), public health, and HI or computer science training. The affirmation of the DoP by reviewers and survey respondents reflects the emergence of a core set of tasks performed and KSs used by informaticians representing a broad spectrum of those currently practicing in the field. CONCLUSION: The HI practice analysis represents the first time that HI professionals have been surveyed to validate a description of their practice. The resulting HI DoP is an important milestone in the maturation of HI as a profession and will inform HI certification, accreditation, and education activities.

Recent Advances in Systems and Network Medicine: Meeting Report from the First International Conference in Systems and Network Medicine.

The First International Conference in Systems and Network Medicine gathered together 200 global thought leaders, scientists, clinicians, academicians, industry and government experts, medical and graduate students, postdoctoral scholars and policymakers. Held at Georgetown University Conference Center in Washington D.C. on September 11-13, 2019, the event featured a day of pre-conference lectures and hands-on bioinformatic computational workshops followed by two days of deep and diverse scientific talks, panel discussions with eminent thought leaders, and scientific poster presentations. Topics ranged from: Systems and Network Medicine in Clinical Practice; the role of -omics technologies in Health Care; the role of Education and Ethics in Clinical Practice, Systems Thinking, and Rare Diseases; and the role of Artificial Intelligence in Medicine. The conference served as a unique nexus for interdisciplinary discovery and dialogue and fostered formation of new insights and possibilities for health care systems advances.

Domains, tasks, and knowledge for clinical informatics subspecialty practice: results of a practice analysis.

OBJECTIVE: The study sought to develop a comprehensive and current description of what Clinical Informatics Subspecialty (CIS) physician diplomates do and what they need to know. MATERIALS AND METHODS: Three independent subject matter expert panels drawn from and representative of the 1695 CIS diplomates certified by the American Board of Preventive Medicine contributed to the development of a draft CIS delineation of practice (DoP). An online survey was distributed to all CIS diplomates in July 2018 to validate the draft DoP. A total of 316 (18.8%) diplomates completed the survey. Survey respondents provided domain, task, and knowledge and skill (KS) ratings; qualitative feedback on the completeness of the DoP; and detailed professional background and demographic information. RESULTS: This practice analysis resulted in a validated, comprehensive, and contemporary DoP comprising 5 domains, 42 tasks, and 139 KS statements. DISCUSSION: The DoP that emerged from this study differs from the 2009 CIS Core Content in 2 respects. First, the DoP reflects the growth in amount, types, and utilization of health data through the addition of a practice domain, tasks, and KS statements focused on data analytics and governance. Second, the final DoP describes CIS practice in terms of tasks in addition to identifying knowledge required for competent practice. CONCLUSIONS: This study (1) articulates CIS diplomate tasks and knowledge used in practice, (2) provides data that will enable the American Board of Preventive Medicine CIS examination to align with current practice, (3) informs clinical informatics fellowship program requirements, and (4) provides insight into maintenance of certification requirements.

The BRIDG project: a technical report.

OBJECTIVES: The Biomedical Research Integrated Domain Group (BRIDG) project is a collaborative initiative between the National Cancer Institute (NCI), the Clinical Data Interchange Standards Consortium (CDISC), the Regulated Clinical Research Information Management Technical Committee (RCRIM TC) of Health Level 7 (HL7), and the Food and Drug Administration (FDA) to develop a model of the shared understanding of the semantics of clinical research. DESIGN: The BRIDG project is based on open-source collaborative principles and an implementation-independent, use-case driven approach to model development. In the BRIDG model, declarative and procedural knowledge are represented using the Unified Modeling Language (UML) class, activity and state diagrams. MEASUREMENTS: The BRIDG model currently contains harmonized semantics from four project use cases: the caXchange project and the patient study calendar project from caBIG; the standard data tabular model (SDTM) from CDISC; and the regulated products submission model (RPS) from HL7. Scalable harmonization processes have been developed to expand the model with content from additional use cases. RESULTS: The first official release of the BRIDG model was published in June 2007. Use of the BRIDG model by the NCI has supported the rapid development of semantic interoperability across applications within the caBIG program. CONCLUSIONS: The BRIDG project has brought together different standards communities to clarify the semantics of clinical research across pharmaceutical, regulatory, and research organizations. Currently, the NCI uses the BRIDG model to support interoperable application development in the caBIG, and CDISC and HL7 are using the BRIDG model to support standards development.

Development of an instrument for measuring clinicians' power perceptions in the workplace.

We report on the development of an instrument to measure clinicians' perceptions of their personal power in the workplace in relation to resistance to computerized physician order entry (CPOE). The instrument is based on French and Raven's six bases of social power and uses a semantic differential methodology. A measurement study was conducted to determine the reliability and validity of the survey. The survey was administered online and distributed via a URL by email to 19 physicians, nurses, and health unit coordinators from a university hospital. Acceptable reliability was achieved by removing or moving some semantic differential word pairs used to represent the six power bases (alpha range from 0.76 to 0.89). The Semantic Differential Power Perception (SDPP) survey validity was tested against an already validated instrument and found to be acceptable (correlation range from 0.51 to 0.81). The SDPP survey instrument was determined to be both reliable and valid.

Five Years of Clinical Informatics Board Certification for Physicians in the United States of America.

OBJECTIVES: To review the highlights of the new Clinical Informatics subspecialty including its history, certification requirements, development of and performance on the certification examination in the United States. METHODS: We reviewed processes for the development of a subspecialty. Data from board certification examinations were collated and analyzed. We discussed eligibility requirements in the fellowship as well as practice pathways. RESULTS: Lessons learned from the development of the Clinical Informatics subspecialty, opportunities, challenges, and future directions for the field are discussed. CONCLUSIONS: There remains a need for fellowship programs and creation and maintenance of a professional home for the subspecialty with the American Medical Informatics Association. Ongoing attention to the currency of the core content is required to maintain an examination designed to test the key concepts within the field of Clinical Informatics.

AMIA Board White Paper: AMIA 2017 core competencies for applied health informatics education at the master's degree level.

This White Paper presents the foundational domains with examples of key aspects of competencies (knowledge, skills, and attitudes) that are intended for curriculum development and accreditation quality assessment for graduate (master's level) education in applied health informatics. Through a deliberative process, the AMIA Accreditation Committee refined the work of a task force of the Health Informatics Accreditation Council, establishing 10 foundational domains with accompanying example statements of knowledge, skills, and attitudes that are components of competencies by which graduates from applied health informatics programs can be assessed for competence at the time of graduation. The AMIA Accreditation Committee developed the domains for application across all the subdisciplines represented by AMIA, ranging from translational bioinformatics to clinical and public health informatics, spanning the spectrum from molecular to population levels of health and biomedicine. This document will be periodically updated, as part of the responsibility of the AMIA Accreditation Committee, through continued study, education, and surveys of market trends.

A systematic review of the performance characteristics of clinical event monitor signals used to detect adverse drug events in the hospital setting.

OBJECTIVE: Despite demonstrated benefits, few healthcare organizations have implemented clinical event monitors to detect adverse drug events (ADEs). The objective of this study was to conduct a systematic review of pharmacy and laboratory signals used by clinical event monitors to detect ADEs in hospitalized adults. DESIGN: We performed a comprehensive search of MEDLINE, CINHAL and EMBASE to identify studies published between 1985 through 2006. Studies were included if they: described a clinical event monitor to detect ADEs in an adult hospital setting; described laboratory or pharmacy ADE signals; and, provided positive predictive values (PPVs) or information to allow the calculation of PPVs for individual ADE signals. MEASUREMENTS: We calculated overall estimates of PPVs and 95% confidence intervals (CIs) for signals reported in 2 or more studies and contained no evidence heterogeneity. Results were examined by signal category: medication levels, laboratory tests, or antidotes. RESULTS: We identified 12 observational studies describing 36 unique ADE signals. Fifteen signals (3 antidotes, 4 medication levels, and 8 laboratory values) contained no evidence of heterogeneity. The pooled PPVs for these individual signals ranged from 0.03 [CI=0.03-0.03] for hypokalemia, to 0.50 [CI=0.39-0.61] for supratherapeutic quinidine level. In general, antidotes (range=0.09-0.11) had the lowest PPVs, followed by laboratory values (0.03-0.27), and medication levels (0.03-0.50). CONCLUSION: Results from this study should help clinical information system and computerized decision support producers develop or improve existing clinical event monitors to detect ADEs in their own hospitals by prioritizing those signals with the highest PPVs [corrected]

User-centered semantic harmonization: a case study.

Semantic interoperability is one of the great challenges in biomedical informatics. Methods such as ontology alignment or use of metadata neither scale nor fundamentally alleviate semantic heterogeneity among information sources. In the context of the Cancer Biomedical Informatics Grid program, the Biomedical Research Integrated Domain Group (BRIDG) has been making an ambitious effort to harmonize existing information models for clinical research from a variety of sources and modeling agreed-upon semantics shared by the technical harmonization committee and the developers of these models. This paper provides some observations on this user-centered semantic harmonization effort and its inherent technical and social challenges. The authors also compare BRIDG with related efforts to achieve semantic interoperability in healthcare, including UMLS, InterMed, the Semantic Web, and the Ontology for Biomedical Investigations initiative. The BRIDG project demonstrates the feasibility of user-centered collaborative domain modeling as an approach to semantic harmonization, but also highlights a number of technology gaps in support of collaborative semantic harmonization that remain to be filled.

How primary care physicians' attitudes toward risk and uncertainty affect their use of electronic information resources.

OBJECTIVE: The research sought to determine if primary care physicians' attitudes toward risk taking or uncertainty affected how they sought information and used electronic information resources when answering simulated clinical questions. METHODS: Using physician-supplied data collected from existing risk and uncertainty scales, twenty-five physicians were classified as risk seekers (e.g., enjoying adventure), risk neutral, or risk avoiders (e.g., cautious) and stressed or unstressed by uncertainty. The physicians then answered twenty-three multiple-choice, clinically focused questions and selected two to pursue further using their own information resources. Think-aloud protocols were used to collect searching process and outcome data (e.g., searching time, correctness of answers, searching techniques). RESULTS: No differences in searching outcomes were observed between the groups. Physicians who were risk avoiding and those who reported stress when faced with uncertainty each showed differences in searching processes (e.g., actively analyzing retrieval, using searching heuristics or rules). Physicians who were risk avoiding tended to use resources that provided answers and summaries, such as Cochrane or UpToDate, less than risk-seekers did. Physicians who reported stress when faced with uncertainty showed a trend toward less frequent use of MEDLINE, when compared with physicians who were not stressed by uncertainty. CONCLUSIONS: Physicians' attitudes towards risk taking and uncertainty were associated with different searching processes but not outcomes. Awareness of differences in physician attitudes may be key in successful design and implementation of clinical information resources.

Effectiveness of clinician-selected electronic information resources for answering primary care physicians' information needs.

OBJECTIVE: To determine if clinician-selected electronic information resources improve primary care physicians' abilities to answer simulated clinical questions. DESIGN: Observational study using hour-long interviews in physician offices and think-aloud protocols. PARTICIPANTS: answered 23 multiple-choice questions and chose 2 to obtain further information using their own information resources. We established which resources physicians chose, processes used, and results obtained when looking for information to support their answers. MEASUREMENTS: Correctness of answers before and after searching, resources used, and searching techniques. RESULTS: 23 physicians sought answers to 46 questions using their own information resources. They spent a mean of 13.0 (SD 5.5) minutes searching for information for the two questions using an average of 1.8 resources per question and a wide variety of searching techniques. On average 43.5% of the answers to the original 23 questions were correct. For the questions that were searched, 18 (39.1%) of the 46 answers were correct before searching. After searching, the number of correct answers was 19 (42.1%). This difference of 1 correct answer was attributed to 6 questions (13.0%) going from an incorrect to correct answer and 5 (10.9%) questions going from a correct to incorrect answer. We found differences in the ability of various resources to provide correct answers. CONCLUSION: For the primary care physicians studied, electronic information resources of choice did not always provide support for finding correct answers to simulated clinical questions and in some instances, individual resources may have contributed to an initially correct answer becoming incorrect.

Creating advanced health informatics certification.

In 2005, AMIA leaders and members concluded that certification of advanced health informatics professionals would offer value to individual practitioners, organizations that hire them, and society at large. AMIA's work to create advanced informatics certification began by leading a successful effort to create the clinical informatics subspecialty for American Board of Medical Specialties board-certified physicians. Since 2012, AMIA has been working to establish advanced health informatics certification (AHIC) for all health informatics practitioners regardless of their primary discipline. In November 2015, AMIA completed the first of 3 key tasks required to establish AHIC, with the AMIA Board of Directors' endorsement of proposed eligibility requirements. This AMIA Board white paper describes efforts to establish AHIC, reports on the current status of AHIC components, and provides a context for the proposed AHIC eligibility requirements.

Eligibility requirements for advanced health informatics certification.

AMIA is leading the effort to strengthen the health informatics profession by creating an advanced health informatics certification (AHIC) for individuals whose informatics work directly impacts the practice of health care, public health, or personal health. The AMIA Board of Directors has endorsed a set of proposed AHIC eligibility requirements that will be presented to the future AHIC certifying entity for adoption. These requirements specifically establish who will be eligible to sit for the AHIC examination and more generally signal the depth and breadth of knowledge and experience expected from certified individuals. They also inform the development of the accreditation process and provide guidance to graduate health informatics programs as well as individuals interested in pursuing AHIC. AHIC eligibility will be determined by practice focus, education in primary field and health informatics, and significant health informatics experience.

Computer decision support as a source of interpretation error: the case of electrocardiograms.

OBJECTIVE: The aim of this study was to determine the effect that the computer interpretation (CI) of electrocardiograms (EKGs) has on the accuracy of resident (noncardiologist) physicians reading EKGs. DESIGN: A randomized, controlled trial was conducted in a laboratory setting from February through June 2001, using a two-period crossover design with matched pairs of subjects randomly assigned to sequencing groups. MEASUREMENTS: Subjects' interpretive accuracy of discrete, cardiologist-determined EKG findings were measured as judged by a board-certified internist. RESULTS: Without the CI, subjects interpreted 48.9% (95% confidence interval, 45.0% to 52.8%) of the findings correctly. With the CI, subjects interpreted 55.4% (51.9% to 58.9%) correctly (p < 0.0001). When the CIs that agreed with the gold standard (Correct CIs) were not included, 53.1% (47.7% to 58.5%) of the findings were interpreted correctly. When the correct CI was included, accuracy increased to 68.1% (63.2% to 72.7%; p < 0.0001). When computer advice that did not agree with the gold standard (Incorrect CI) was not provided to the subjects, 56.7% (48.5% to 64.5%) of findings were interpreted correctly. Accuracy dropped to 48.3% (40.4% to 56.4%) when the incorrect computer advice was provided (p = 0.131). Subjects erroneously agreed with the incorrect CI more often when it was presented with the EKG 67.7% (57.2% to 76.7%) than when it was not 34.6% (23.8% to 47.3%; p < 0.0001). CONCLUSIONS: Computer decision support systems can generally improve the interpretive accuracy of internal medicine residents in reading EKGs. However, subjects were influenced significantly by incorrect advice, which tempers the overall usefulness of computer-generated advice in this and perhaps other areas.

Medication error reporting in long term care.

BACKGROUND: Medication errors are common causes of medical error in the long-term care (LTC) setting. Despite their frequency and potential clinical impact, most medication errors in LTC facilities remain unreported. Before better reporting systems can be developed to reduce clinically significant medication errors, it is necessary to understand how current medication error reporting systems function. OBJECTIVE: This study describes the medication use and medication error reporting processes, and characterizes the knowledge, attitudes, and beliefs about medication errors of the nursing staff at a single LTC facility. METHODS: Three methods were used to characterize the medication use and medication error reporting processes and the nursing staff's perceptions about such errors. First, key elements and basic processes were defined through observation and semi-structured interviews. Second, medication error reports were reviewed and summarized over a 21-month period. Third, nursing facility staff were surveyed about their knowledge, attitudes, and beliefs concerning medication errors. RESULTS: The medication use process in the LTC setting is similar to that employed in the acute care setting, consisting of 5 steps: prescribing, documenting, dispensing, administering, and monitoring. In the facility studied, an average of 4.7 medication error reports were submitted per month. Staff felt that half of all medication errors were identified and communicated informally through change-of-shift reports rather than through medication error reports. Most staff (85%) believed that disciplinary action was taken against the person who committed an error. CONCLUSIONS: The medication error policies and processes of the LTC facility studied were associated with a low frequency of formal reporting, a narrow perspective on the sources of error, and concerns about disciplinary action. Research is needed to better identify errors, develop interventions that broaden the monitoring perspective to include all health care professionals, reduce the work of reporting, standardize the information collected, and create an institutional atmosphere of participation rather than punishment.

Development and evaluation of a charge capture program for long-term care providers.

OBJECTIVE: Clinicians often have difficulty determining the appropriate Current Procedural Terminology Evaluation and Management code to assign to the type and intensity of patient care they provide. The purpose of this study was to develop, implement, and evaluate a handheld charge capture program for use by providers in the long-term care setting. DESIGN: Using a pre-post study design, we compared the coding accuracy and user satisfaction of an established paper process with a handheld charge capture program created for this study by means of: (1) preimplementation and postimplementation assessment of coding accuracy, and (2) preimplementation and postimplementation clinician survey. SETTING: We studied an academic division of geriatric medicine. PARTICIPANTS: Participants consisted of six clinicians who currently spend at least 50% of their clinical time practicing in the long-term care setting. INTERVENTION: A handheld charge capture program to replace the current paper-based charge capture process was reviewed. RESULTS: Overall coding accuracy improved by approximately 20% when the handheld program was used instead of a paper coding process. The majority of clinicians found that the handheld program was more widely available, efficient, easier to use, and encouraged the participants to document more completely and accurately in the patient's medical record. CONCLUSION: A handheld billing and coding program used by clinicians who provide care for long-term care residents is not only feasible, but leads to an improvement in coding accuracy when compared with a paper process. In addition, clinician satisfaction toward the billing and coding processes improved with the use of the handheld program.

Predicting the impact of a new information system on workflow using a discrete event simulation.

Computer simulation of an information system prior to its implementation can predict time and workflow changes in a hospital department, while offering a common ground of communication across various levels in the organization. Often, the simulation can predict unexpected effects of changes to the work environment and allow experimentation with alternative scenarios at minimal cost to the department or the organization. In this paper we describe a discrete-event simulation experiment that predicted an unexpected increase in routine specimen processing time with the introduction of an information system in the HLA tissue typing lab at a major transplant center. The computer simulation enabled the reallocation of existing staff prior to the system implementation.