Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study.

BACKGROUND: The purpose of this study was to identify recommended practices for computerized clinical decision support (CDS) development and implementation and for knowledge management (KM) processes in ambulatory clinics and community hospitals using commercial or locally developed systems in the U.S. METHODS: Guided by the Multiple Perspectives Framework, the authors conducted ethnographic field studies at two community hospitals and five ambulatory clinic organizations across the U.S. Using a Rapid Assessment Process, a multidisciplinary research team: gathered preliminary assessment data; conducted on-site interviews, observations, and field surveys; analyzed data using both template and grounded methods; and developed universal themes. A panel of experts produced recommended practices. RESULTS: The team identified ten themes related to CDS and KM. These include: 1) workflow; 2) knowledge management; 3) data as a foundation for CDS; 4) user computer interaction; 5) measurement and metrics; 6) governance; 7) translation for collaboration; 8) the meaning of CDS; 9) roles of special, essential people; and 10) communication, training, and support. Experts developed recommendations about each theme. The original Multiple Perspectives framework was modified to make explicit a new theoretical construct, that of Translational Interaction. CONCLUSIONS: These ten themes represent areas that need attention if a clinic or community hospital plans to implement and successfully utilize CDS. In addition, they have implications for workforce education, research, and national-level policy development. The Translational Interaction construct could guide future applied informatics research endeavors.

Computerized provider order entry adoption: implications for clinical workflow.

OBJECTIVE: To identify and describe unintended adverse consequences related to clinical workflow when implementing or using computerized provider order entry (CPOE) systems. METHODS: We analyzed qualitative data from field observations and formal interviews gathered over a three-year period at five hospitals in three organizations. Five multidisciplinary researchers worked together to identify themes related to the impacts of CPOE systems on clinical workflow. RESULTS: CPOE systems can affect clinical work by 1) introducing or exposing human/computer interaction problems, 2) altering the pace, sequencing, and dynamics of clinical activities, 3) providing only partial support for the work activities of all types of clinical personnel, 4) reducing clinical situation awareness, and 5) poorly reflecting organizational policy and procedure. CONCLUSIONS: As CPOE systems evolve, those involved must take care to mitigate the many unintended adverse effects these systems have on clinical workflow. Workflow issues resulting from CPOE can be mitigated by iteratively altering both clinical workflow and the CPOE system until a satisfactory fit is achieved.

The extent and importance of unintended consequences related to computerized provider order entry.

BACKGROUND: Computerized provider order entry (CPOE) systems can help hospitals improve health care quality, but they can also introduce new problems. The extent to which hospitals experience unintended consequences of CPOE, which include more than errors, has not been quantified in prior research. OBJECTIVE: To discover the extent and importance of unintended adverse consequences related to CPOE implementation in U.S. hospitals. DESIGN, SETTING, AND PARTICIPANTS: Building on a prior qualitative study involving fieldwork at five hospitals, we developed and then administered a telephone survey concerning the extent and importance of CPOE-related unintended adverse consequences to representatives from 176 hospitals in the U.S. that have CPOE. MEASUREMENTS: Self report by key informants of the extent and level of importance to the overall function of the hospital of eight types of unintended adverse consequences experienced by sites with inpatient CPOE. RESULTS We found that hospitals experienced all eight types of unintended adverse consequences, although respondents identified several they considered more important than others. Those related to new work/more work, workflow, system demands, communication, emotions, and dependence on the technology were ranked as most severe, with at least 72% of respondents ranking them as moderately to very important. Hospital representatives are less sure about shifts in the power structure and CPOE as a new source of errors. There is no relation between kinds of unintended consequences and number of years CPOE has been used. Despite the relatively short length of time most hospitals have had CPOE (median five years), it is highly infused, or embedded, within work practice at most of these sites. CONCLUSIONS: The unintended consequences of CPOE are widespread and important to those knowledgeable about CPOE in hospitals. They can be positive, negative, or both, depending on one's perspective, and they continue to exist over the duration of use. Aggressive detection and management of adverse unintended consequences is vital for CPOE success.

A survey of U.S.A. acute care hospitals' computer-based provider order entry system infusion levels.

We developed and fielded a survey to help clinical information system designers, developers, and implementers better understand the infusion level, or the extent and sophistication of CPOE feature availability and use by clinicians within acute care hospitals across the United States of America. In the 176 responding hospitals, we found that CPOE had been in place a median of 5 years and that the median percentage of orders entered electronically was 90.5%. Greater than 96% of the sites used CPOE to enter pharmacy, laboratory and imaging orders; 82% were able to access all aspects of the clinical information system with a single sign-on; 86% of the respondents had order sets, drug-drug interaction warnings, and pop-up alerts even though nearly all hospitals were community hospitals with commercial systems; and 90% had a CPOE committee with a clinician representative in place. While CPOE has not been widely adopted after over 30 years of experimentation, there is still much that can be learned from this relatively small number of highly infused (with CPOE and clinical decision support) organizations.

Types of unintended consequences related to computerized provider order entry.

OBJECTIVE: To identify types of clinical unintended adverse consequences resulting from computerized provider order entry (CPOE) implementation. DESIGN: An expert panel provided initial examples of adverse unintended consequences of CPOE. The authors, using qualitative methods, gathered and analyzed additional examples from five successful CPOE sites. METHODS: Using a card sort method, the authors developed a categorization scheme for the 79 unintended consequences initially identified and then iteratively modified the scheme to categorize 245 additional adverse consequences resulting from fieldwork. Because the focus centered on consequences requiring prevention or remedial action, the authors did not further analyze reported unintended beneficial (positive) consequences. RESULTS: Unintended adverse consequences (UACs) fell into nine major categories (in order of decreasing frequency): 1) more/new work for clinicians; 2) unfavorable workflow issues; 3) never ending system demands; 4) problems related to paper persistence; 5) untoward changes in communication patterns and practices; 6) negative emotions; 7) generation of new kinds of errors; 8) unexpected changes in the power structure; and 9) overdependence on the technology. Clinical decision support features introduced many of these unintended consequences. CONCLUSION: Identifying and understanding the types and in some instances the causes of unintended adverse consequences associated with CPOE will enable system developers and implementers to better manage implementation and maintenance of future CPOE projects.

A survey of factors affecting clinician acceptance of clinical decision support.

BACKGROUND: Real-time clinical decision support (CDS) integrated into clinicians' workflow has the potential to profoundly affect the cost, quality, and safety of health care delivery. Recent reports have identified a surprisingly low acceptance rate for different types of CDS. We hypothesized that factors affecting CDS system acceptance could be categorized as relating to differences in patients, physicians, CDS-type, or environmental characteristics. METHODS: We conducted a survey of all adult primary care physicians (PCPs, n = 225) within our group model Health Maintenance Organization (HMO) to identify factors that affect their acceptance of CDS. We defined clinical decision support broadly as "clinical information" that is either provided to you or accessible by you, from the clinical workstation (e.g., enhanced flow sheet displays, health maintenance reminders, alternative medication suggestions, order sets, alerts, and access to any internet-based information resources). RESULTS: 110 surveys were returned (49%). There were no differences in the age, gender, or years of service between those who returned the survey and the entire adult PCP population. Overall, clinicians stated that the CDS provided "helps them take better care of their patients" (3.6 on scale of 1:Never-5:Always), "is worth the time it takes" (3.5), and "reminds them of something they've forgotten" (3.2). There was no difference in the perceived acceptance rate of alerts based on their type (i.e., cost, safety, health maintenance). When asked about specific patient characteristics that would make the clinicians "more", "equally" or "less" likely to accept alerts: 41% stated that they were more (8% stated "less") likely to accept alerts on elderly patients (> 65 yrs); 38% were more (14% stated less) likely to accept alerts on patients with more than 5 current medications; and 38% were more (20% stated less) likely to accept alerts on patients with more than 5 chronic clinical conditions. Interestingly, 80% said they were less likely to accept alerts when they were behind schedule and 84% of clinicians admitted to being at least 20 minutes behind schedule "some", "most", or "all of the time". CONCLUSION: Even though a majority of our clinical decision support suggestions are not explicitly followed, clinicians feel they are of benefit and would be even more beneficial if they had more time available to address them.

Adding insight: a qualitative cross-site study of physician order entry.

The research questions, strategies, and results of a 7-year qualitative study of computerized physician order entry implementation (CPOE) at successful sites are reviewed over time. The iterative nature of qualitative inquiry stimulates a consecutive stream of research foci, which, with each iteration, add further insight into the overarching research question. A multidisciplinary team of researchers studied CPOE implementation in four organizations using a multi-method approach to address the question "what are the success factors for implementing CPOE?" Four major themes emerged after studying three sites; ten themes resulted from blending the first results with those from a fourth site; and twelve principles were generated when results of a qualitative analysis of consensus conference transcripts were combined with the field data. The study has produced detailed descriptions of factors related to CPOE success and insight into the implementation process.

Adding insight: a qualitative cross-site study of physician order entry.

The research questions, strategies, and results of a six-year qualitative study of computerized physician order entry implementation (CPOE) at successful sites are reviewed over time. The iterative nature of qualitative inquiry stimulates a consecutive stream of research foci which, with each iteration, add further insight into the overarching research question. A multidisciplinary team of researchers studied CPOE implementation in four organizations using a multi-method approach to address the question "what are the success factors for implementing CPOE?" Four major themes emerged after studying three sites; ten themes resulted from blending the first results with those from a fourth site; and twelve principles were generated when results of a qualitative analysis of consensus conference transcripts were combined with the field data. The study has produced detailed descriptions of factors related to CPOE success and insight into the implementation process.

Multiple perspectives on the meaning of clinical decision support.

Clinical Decision Support (CDS) is viewed as a means to improve safety and efficiency in health care. Yet the lack of a consensus around what is meant by CDS represents a barrier to effective design, use, and utilization of CDS tools. We conducted a multi-site qualitative inquiry to understand how different people define and describe CDS. Using subjects' multiple perspectives we were able to gain new insights as to what stakeholders want CDS to achieve and how to achieve it; even at times when those perspectives are competing and conflicting.

Multiple perspectives on the meaning of clinical decision support.

Clinical Decision Support (CDS) is viewed as a means to improve safety and efficiency in health care. Yet the lack of consensus about what is meant by CDS represents a barrier to effective design, implementation, and utilization of CDS tools. We conducted a multi-site qualitative inquiry to understand how different people define and describe CDS. Using subjects' multiple perspectives we were able to gain new insights as to what stakeholders want CDS to achieve and how to achieve it even when those perspectives are competing and conflicting.

Persistent paper: the myth of "going paperless".

How does paper usage change following the introduction of Computerized Physician Order Entry and the Electronic Medical Record (EMR/CPOE)? To answer that question we analyzed data collected from fourteen sites across the U.S. We found paper in widespread use in all institutions we studied. Analysis revealed psychological, ergonomic, technological, and regulatory reasons for the persistence of paper in an electronic environment. Paper has unique attributes allowing it to fill gaps in information timeliness, availability, and reliability in pursuit of improved patient care. Creative uses have led to "better paper."

Assessing the anticipated consequences of Computer-based Provider Order Entry at three community hospitals using an open-ended, semi-structured survey instrument.

OBJECTIVE: To determine what "average" clinicians in organizations that were about to implement Computer-based Provider Order Entry (CPOE) were expecting to occur, we conducted an open-ended, semi-structured survey at three community hospitals. METHODS: We created an open-ended, semi-structured, interview survey template that we customized for each organization. This interview-based survey was designed to be administered orally to clinicians and take approximately 5 min to complete, although clinicians were allowed to discuss as many advantages or disadvantages of the impending system roll-out as they wanted to. RESULTS: Our survey findings did not reveal any overly negative, critical, problematic, or striking sets of concerns. However, from the standpoint of unintended consequences, we found that clinicians were anticipating only a few of the events, emotions, and process changes that are likely to result from CPOE. CONCLUSIONS: The results of such an open-ended survey may prove useful in helping CPOE leaders to understand user perceptions and predictions about CPOE, because it can expose issues about which more communication, or discussion, is needed. Using the survey, implementation strategies and management techniques outlined in this paper, any chief information officer (CIO) or chief medical information officer (CMIO) should be able to adequately assess their organization's CPOE readiness, make the necessary mid-course corrections, and be prepared to deal with the currently identified unintended consequences of CPOE should they occur.

Some unintended consequences of clinical decision support systems.

Clinical decision support systems (CDS) coupled with computerized physician/provider order entry (CPOE) can improve the quality of patient care and the efficiency of hospital operations. However, they can also produce unintended consequences. Using qualitative methods, a multidisciplinary team gathered and analyzed data about the unintended consequences of CPOE, identifying nine types, and found that CDS-generated unintended consequences appeared among all types. Further analysis of 47 CDS examples uncovered three themes related to CDS content: elimination or shifting of human roles; difficulty in keeping content current; and inappropriate content. Three additional themes related to CDS presentation were found: rigidity of the system; alert fatigue; and potential for errors. Management of CDS must include careful selection and maintenance of content and prudent decision making about human computer interaction opportunities.

Overdependence on technology: an unintended adverse consequence of computerized provider order entry.

Computerized provider order entry (CPOE) and other clinical information systems can help reduce medical errors, promote practice standardization, and improve the quality of patient care. However, implementing these systems can result in unintended adverse consequences. Our multidisciplinary team used qualitative methods to gather and analyze data describing unintended adverse consequences related to CPOE adoption and use. Overdependence on technology emerged as one of nine major types we identified. Careful analysis of these data revealed three themes: 1) system downtime can create chaos when there are insufficient backup systems in place, 2) users have false expectations regarding data accuracy and processing, and 3) some clinicians cannot work efficiently without computerized systems. We provide recommendations for mitigating these important issues.

Categorizing the unintended sociotechnical consequences of computerized provider order entry.

OBJECTIVE: To describe the kinds of unintended consequences related to the implementation of computerized provider order entry (CPOE) in the outpatient setting. DESIGN: Ethnographic and interview data were collected by an interdisciplinary team over a 7 month period at four clinics. MEASUREMENTS: Instances of unintended consequences were categorized using an expanded Diffusion of Innovations theory framework. RESULTS: The framework was clarified and expanded. There are both desirable and undesirable unintended consequences, and they can be either direct or indirect, but there are also many consequences that are not clearly either desirable or undesirable or may even be both, depending on one's perspective. The undesirable consequences include error and security concerns and issues related to alerts, workflow, ergonomics, interpersonal relations, and reimplementations. CONCLUSION: Consequences of implementing and reimplementing clinical systems are complex. The expanded Diffusion of Innovations theory framework is a useful tool for analyzing such consequences.

An unintended consequence of CPOE implementation: shifts in power, control, and autonomy.

Having found that an unintended consequence of computerized provider order entry (CPOE) implementation is "changes in the power structure" of the organization, we sought a deeper understanding of what was happening and why. If such consequences can be anticipated, they can be better managed. Using qualitative methods to study five successful CPOE sites, a multidisciplinary team found that CPOE enables shifts in power related to work redistribution and safety initiatives and causes a perceived loss of control and autonomy by clinicians. With recognition of the extent of these shifts, clinicians can anticipate them and will no longer be surprised by them. Greater provider involvement in planning, quality initiatives, and the work of clinical information coalitions/committees can benefit the organization and provide a different kind of power and satisfaction to clinicians.

Ambulatory computerized physician order entry implementation.

As part of broader effort to identify success factors for implementing computerized physician order entry(CPOE), factors specific to the ambulatory setting were investigated in the field at Kaiser Permanente Northwest. A multidisciplinary team of five qualitative researchers spent seven months at four clinics conducting observations, interviews, and focus groups. The team analyzed the data using a combination of template and grounded theory approaches. The result is a description of fourteen themes, clustered into technology, organizational,personal, and environmental categories. While similar to inpatient study results in many respects,this outpatient CPO investigation generated subtly different themes.